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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE = j 



Transmitted herewith for filing is the Patent Application of: Larry S. Marso, et al. 

For: METHOD AND SYSTEM FOR RATING USERS, USER-CONTRIBUTED ITEMS, 

GROUPINGS OF RELATED USER-CONTRIBUTED ITEMS AND OTHER ITEMS 

ENCLOSURES 

[X] New Application Transmittal; 

£<] 79-page application including specification, claims and abstract; 
^[ 36 sheets of informal drawings; 

[X] A postcard for return to us as proof of receipt of the above documents. 

TYPE OF FILING 

Q This application claims the benefit of an earlier filed U.S. Patent Application under 35 USC 
120. 

[X] Please accord Applicant the benefit of the priority date of November 26, 1999 to this case 
pursuant to 35 USC 119. Applicant's claim for priority is based on provisional patent 
application serial no. 60/167,594 filed on said date. 

£<] This is an application filed pursuant to 37 CFR 1 .53, permitting receipt of a filing date upon 
filing of specification, claims and drawings, if required, with applicant being given a period 
of one month from the date of notice to file the fee and oath or declaration. 

[X] In the event any parts of this application are incomplete, please treat this as a filing under 
37 CFR 1.53 as defined just above. 



CERTIFICATE OF MAILING BY "EXPRESS MAIL": I hereby certify that this correspondence is being deposited with the U.S. Postal Service 
as Express Mail No. EL596839628US addressed to the Commissioner of Patents ^.Trademarks, Washington, D.C. 20231 on November 27, 2000 
by Diana Deanng. /J ^ f\ 



Commissioner of Patents and Trademarks 
Washington, D.C. 20231 



FILING TRANSMITTAL 



Attorney's Docket No. : 32803-0269*596 




CERTIFICATE OF MAILING 



Date. November 27, 2000 
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FEE CALCULATION 

The filing fee has been calculated as shown below: 

OTHER THAN A 





SMALL ENTITY 


OR 


SMALL ENTITY 






BASIC FEE Design Patent 


$155 


$ 




$310 


$ 




BASIC FEE Utility Patent 


$345 


$ 




$690 


$ 




EXTRA FEES 




RATE FEE 




RATE FEE 






TOTAL CLAIMS 


MINUS 20 = 




x 11 = 


$ 


x22 = 


S 


INDEP. CLAIMS 


MINUS 3 = 




x40 = 


$ 


x80 = 


s 


□ MULTIPLE DEP. CLAIM 




+130 = 


$ 


+260 = 


$ 


□ ASSIGNMENT 






+ 40 = 


$ 


+ 40 = 


$ 


□ RULE 53 SURCHARGE 






+ 65 = 


$ 


+130 = 


$ 



TOTAL $ $ 



FEE PAYMENT 

[~j Attached is Check No. in the sum of $ to cover the filing fee. 
□ Please charge Account No. the sum of $ 

FEE DEFICIENCY 

| | The Commissioner is authorized to charge (or credit any overpayment) to deposit account 
No. : 

[~J Any additional filing fees required under 37 CFR 1.16, except Rule 53 filings, which 
will be paid within the time permitted by PTOL 1533. 

[ | Assignment Recordal fees. 

| | The filing fee and surcharge under 37 CFR 1.16, patent application processing fees under 
37 CFR 1.17 and patent issue fees under 37 CFR 1.18 are intended to be paid by our firm 
as they arise. As no abandonment is intended by any inadvertent nonpayment of fees, the 
Commissioner is hereby authorized to charge payment of such fees as from time to time 
come due, if not paid prior to due date to our Deposit Account No. 

| | A duplicate copy of this sheet is enclosed. 

Respectfully submitted, 



Dated: November 27, 2000 

PILLSBURY MADISON & SUTRO LLP 
2550 Hanover Street 
Palo Alto, CA 94304-1115 
Telephone: (650)233-4510 
Facsimile: (650) 233-4545 
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Practitioner's Docket No. 32803-0269596 



PATENT 



Preliminary Classification: 
Proposed Class: 
Subclass: 

NOTE: "All applicants are requested to include a preliminary classification on newly filed patent 

applications. The preliminary classification, preferably class and subclass designations, should be 
identified in the upper nght-hand comer of the letter of transmittal accompanying the application 
papers, for example 'Proposed Class 2, subclass 129.' " M.P.E.P. § 601, 7th ed. 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

Box Patent Application 

Assistant Commissioner for Patents 

Washington, D.C. 20231 

NEW APPLICATION TRANSMITTAL 

Transmitted herewith for filing is the patent application of 
Inventor(s): Larry S. Mar'so and Brian E, Litzinger 



WARNING: 37 C.F.R. § 1.41(a)(1) points out: 

"(a) A patent is applied for in the name or names of the actual inventor or inventors. 

"(1) The inventorship of a nonprovisional application is that inventorship set forth in the oath or 
declaration as prescribed by § 1.63, except as provided for in § 1.53(d)(4) and § 1.63(d). If an 
oath or declaration as prescribed by § 1.63 is not filed during the pendency of a nonprovisional 
application, the inventorship is that inventorship set forth in the application papers filed pursuant 
to § 1.53(b), unless a petition under this paragraph accompanied by the fee set forth in § 1.1 7(i) 
is filed supplying or changing the name or names of the inventor or inventors." 

For (title): METHOD AND SYSTEM FOR RATING USERS, USER-CONTRIBUTED ITEMS, 
GROUPINGS OF RELATED USER-CONTRIBUTED ITEMS AND OTHER ITEMS 



CERTIFICATION UNDER 37 C.F.R. § 1.10* 
(Express Mail label number is mmndmtory.) 
(Express Mail certification is optional.) 

I hereby certify that this New Application Transmittal and the documents referred to as attached therein are being 

deposited with the United States Postal Service on this date / 7 ~ & Q, in an envelope 

as "Express Mail Post Office to Addressee," mailing Label Number EL596839628US ad- 
dressed to the: Assistant Commissioner for Patents, Washington, D.C. 20231 . 

Diana Dearing 



(type or print name of person mailing paper) 
gnature of person mailing paper / 



Signature c 

WARNING: Certificate of mailing (first class) or facsimile transmission procedures of 37 C.F.R. § 1.8 cannot be 
used to obtain a date of mailing or transmission for this correspondence. 

^WARNING: Each paper or fee filed by "Express Mail" must have the number of the "Express Mail" mailing label 
placed thereon prior to mailing. 37 C.F.R. § 1.10(b). 

"Since the filing .of correspondence under § 1.10 without the Express Mail mailing label thereon 
is an oversight that can be avoided by the exercise of reasonable care, requests for waiver of this 
requirement will not be granted on petition. " Notice of Oct. 24, 1996, 60 Fed. Reg. 56,439, at 56, 442. 

(New Application Transmittal [4-1] — page 1 of 11) 



1. Type of Application 

This new application is for a(n) 

(check one applicable item below) 

3 Original (non provisional) 

□ Design 
□ Plant 

WARNING: Do not use this transmittal for a completion in the U.S. of an International Application under 35 
U.S.C. § 371(c)(4), unless the International Application is being filed as a divisional, continuation 
or continuation-in-part application. 

WARNING: Do not use this transmittal for the filing of a provisional application. 

NOTE: If one of the following 3 items apply, then complete and attach ADDED PAGES FOR NEW APPLICATION 
TRANSMITTAL WHERE BENEFIT OF A PRIOR U.S. APPLICATION CLAIMED and a NOTIFICATION 
IN PARENT APPLICATION OF THE FILING OF THIS CONTINUATION APPLICATION. 

□ Divisional. 

□ Continuation. 

□ Continuation-in-part (C-l-P). 

2. Benefit of Prior U.S. Application(s) (35 U.S.C. §§ 119(e), 120, or 121) 

NOTE: A nonprovisional application may claim an invention disclosed in one or more pnor filed copending 
nonprovisional applications or copending international applications designating the United States of 
America. In order for a nonprovisional application to claim the benefit of a prior filed copending 
nonprovisional application or copending international application designating the United States of 
America, each prior application must name as an inventor at least one inventor named in the later filed 
nonprovisional application and disclose the named inventor's invention claimed in at least one claim 
of the later filed nonprovisional application in the manner provided by the first paragraph of 35 U.S.C. 
§ 1 12. Each prior application must also be: 

(i) An international application entitled to a filing date in accordance with PCT Article 1 1 and 
designating the United States of Amenca: or 

fii) Complete as set forth in § 1.51(b); or 

(iii) Entitled to a filing date as set forth in § 1.53(b) or § 1.53(d) and include the basic filing fee set 
forth in § 1. 16; or 

(iv) Entitled to a filing date as set forth in § 1.53(b) and have paid therein the processing and retention 
fee set forth in § 1.21(1) within the time penod set forth in § 1.53(f). 

37 C.F.R. § 1.78(a)(1). 

NOTE: If the new application being transmitted is a divisional, continuation or a continuation-in-part of a parent 
case, or where the parent case is an International Application which designated the U.S., or benefit 
of a prior provisional application is claimed, then check the following item and complete and attach 
ADDED PAGES FOR NEW APPLICATION TRANSMITTAL WHERE BENEFIT OF PRIOR U.S. APPLICA- 
TION^) CLAIMED. 

WARNING: If an application claims the benefit of the filing date of an earlier filed application under 35 U.S.C. 

§§ 120, 121 or 365(c). the 20-year term of that application will be based upon the filing date of 
the earliest U.S. application that the application makes reference to under 35 U.S.C. §§ 120, 121 
or 365(c). (35 U.S.C. § 154(a)(2) does not take into account, for the determination of the patent 
term, any application on which pnorityis claimed under 35 U.S.C. §§ 119, 365(a) or 365(b).) For 
a c-i-p application, applicant should review whether any claim in the patent that will issue is 
supported by an earlier application and, if not, the applicant should consider canceling the reference 
to the earlier filed application. The term of a patent is not based on a claim-by-claim approach. 
See Notice of April 14, 1995, 60 Fed. Reg. 20, 195, at 20,205. 
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WARNING: When the last day of pendency of a provisional application fails on a Saturday, Sunday, or Federal 
holiday within the District of Columbia, any nonprovisional application claiming benefit of the 
provisional application must be filed prior to the Saturday, Sunday, or Federal holiday within the 
District of Columbia. See 37 C.F.R. § 1.78(a)(3). 

GS The new appiication being transmitted claims the benefit of prior U.S. applica- 
tions). Enclosed are ADDED PAGES FOR NEW APPLICATION TRANSMITTAL 
WHERE BENEFIT OF PRIOR U.S. APPLICATION(S) CLAIMED. 

3. Papers Enclosed 

A. Required for filing date under 37 C.F.R. § 1.53(b) (Regular) or 37 C.F.R. § 1.153 
(Design) Application 

Pages of specification 

5_ Pages of claims 

36 Sheets of drawing 

WARNING: DO NOT submit onginal drawings. A high quality copy of the drawings should be supplied when 
filing a patent application. The drawings that are submitted to the Office must be on strong, white, 
smooth, and non-shiny paper and meet the standards according to § 1.84. If corrections to the 
drawings are necessary, they should be made to the original drawing and a high-quality copy of 
the corrected original drawing then submitted to the Office. Only one copy is required or desired. 
For comments on proposed then-new 37 C.F.R. § 1.84, see Notice of March 9, 1988 (1990 O.G. 
57-62). 

NOTE: "Identifying indicia, if provided, should include the application number or the title of the invention, 
inventor's name, docket number (if any), and the name and telephone number of a person to call if 
the Office is unable to match the drawings to the proper application. This information should be placed 
on the back of each sheet of drawing a minimum distance of 1.5 cm. (5/8 inch) down from the top 
of the page . . ."37 C.F.R. § 1.84(c)). 

(complete the following, if applicable) 

□ The enclosed drawing(s) are photograph(s), and there is also attached a 
"PETITION TO ACCEPT PHOTOGRAPH(S) AS DRAWING(S)." 37 C.F.R. 
§ 1.84(b). 

□ formal 

□ informal 

B. Other Papers Enclosed 

Pages of declaration and power of attorney 

/_ Pages of abstract 

Other 

4. Additional papers enclosed 

□ Amendment to claims 

□ Cancel in this applications claims before 

calculating the filing fee. (At least one original independent claim must be 
retained for filing purposes.) 

□ Add the claims shown on the attached amendment. (Claims added have 
been numbered consecutively following the highest numbered original 
claims.) 

□ Preliminary Amendment 

□ Information Disclosure Statement (37 C.F.R. § 1.98) 

□ Form PTO-1449 (PTO/SB/08A and 08B) 

□ Citations 



□ Declaration of Biological Deposit 

□ Submission of "Sequence Listing," computer readable copy and/or amendment 
pertaining thereto for biotechnology invention containing nucleotide and/or 
amino acid sequence. 

□ Authorization of Attorney(s) to Accept and Follow Instructions from Representa- 
tive 

□ Special Comments 

□ Other 

5. Declaration or oath (including power of attorney) 

NOTE: A newly executed declaration is not required in a continuation or divisional application provided that 
the prior nonprovisional application contained a declaration as required, the application being filed is 
by all or fewer than all the inventors named in the pnor application, there is no new matter in the 
application being filed, and a copy of the executed declaration filed in the prior application (showing 
the signature or an indication thereon that it was signed) is submitted. The copy must be accompanied 
by a statement requesting deletion of the names of personfs) who are not inventors of the application 
being filed. If the declaration in the prior application was filed under § 1.47, then a copy of that 
declaration must be filed accompanied by a copy of the decision granting § 1.47 status or, if a nonsigning 
person under § 1.47 has subsequently joined in a prior application, then a copy of the subsequently 
executed declaration must be filed. See 37 C.F.R. §§ 1 .63(d)(1 H3). 

NOTE: A declaration filed to complete an application must be executed, identify the specification to which it 
is directed, identify each inventor by full name including family name and at least one given name, without 
abbreviation together with any other given name or initial, and the residence, post office address and 
country or citizenship of each inventor, and state whether the inventor is a sole or joint inventor. 37 
C.F.R. § 1.63<a)(1H4)- 

NOTE: "The inventorship of a nonprovisional application is that inventorship set forth in the oath or declaration 
as prescribed by § 1.62, except as provided for in § 1.53(d)(4) and § 1.63(d). If an oath or declaration 
as prescnbed by § 1.63 is not filed during the pendency of a nonprovisional application, the inventorship 
is that inventorship set forth in the application papers filed pursuant to § 1.53(b), unless a petition under 
this paragraph accompanied by the fee set forth in § 1.17(f) is filed supplying or changing the name 
or names of the inventor or inventors." 37 C.F.R. § 1.41(a)(1). 

Enclosed .(uauiyunl) 
Executed by 

(check ail applicable boxes) 

□ inventor(s). 

□ legal representative of inventors). 
37 C.F.R. §§ 1.42 or 1.43. 

□ joint inventor or person showing a proprietary 
interest on behalf of inventor who refused to sign 
or cannot be reached. 

□ This is the petition required by 37 C.F.R. § 1 .47 and the statement 
required by 37 C.F.R. § 1.47 is also attached. See item 13 below 
for fee. 
EF / ^Not Enclosed. 

NOTE: Where the filing is a completion in the U.S. of an International Application or where the completion of 
the U.S. application contains subject matter in addition to the International Application, the application 
may be treated as a continuation or continuation-in-part, as the case may be, utilizing ADDED PAGE 
FOR NEW APPLICATION TRANSMITTAL WHERE BENEFIT OF PRIOR U.S. APPLICATION CLAIMED. 

□ Application is made by a person authorized under 37 C.F.R. § 1.41(c) on 
behalf of all the above named inventor(s). 
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(The declaration or oath, along with the surcharge required by 37 C.F.R § 1.16(e) 
can be filed subsequently). 

□ Showing that the filing is authorized. 

(not required unless called into question. 37 C.F.R. § 1.41(d)) 

6. Inventorship Statement 

WARNING: If the named inventors are each not the inventors of all the claims an explanation, including the 
ownership of the various claims at the time the last claimed invention was made, should be 
submitted. 

The inventorship for all the claims in this application are: 
0 The same. 

or 

□ Not the same. An explanation, including the ownership of the various claims at 
the time the last claimed invention was made, 

□ is submitted. 

□ will be submitted. 

7. Language 

NOTE; An application including a signed oath or declaration may be filed in a language other than English. 
An English translation of the non-English language application and the processing fee of $130.00 
required by 37 C.F.R. § l.17(k) is required to be filed with the application, or within such time as may 
be set by the Office. 37 C.F.R. § 1.52(d). 

English 

□ Non-English 

□ The attached translation includes a statement that the translation is accu- 
rate. 37 C.F.R § 1.52(d). 

8. Assignment 

U An assignment of the invention to High Regard, Inc. : 



□ is attached. A separate □ "COVER SHEET FOR ASSIGNMENT (DOCU- 
MENT) ACCOMPANYING NEW PATENT APPLICATION" or □ FORM PTO 
1595 is also attached. 

(3 will follow. 

NOTE: "If an assignment is submitted with a new application, send two separate letters-one for the application 

and one for the assignment." Notice of May 4, 1990 (1114 O.G. 77-78). 
WARNING: A newly executed 'CERTIFICATE UNDER 37 C.F.R. § 3. 73(b)" must be filed when a continuation- 
in-part application is filed by an assignee. Notice of April 30, 1993, 1150 O.G. 62-64. 
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9. Certified Copy 

Certified copy(ies) of application(s) 



Country 


Appln. No. 


Filed 


Country 


Appln. No. 


Filed 


Country 


Appln. No. 


Filed 



from which priority is claimed 

□ is (are) attached. 

□ will follow. 

NOTE: The foreign application forming the basis for the claim for prionty must be referred to tn the oath or 

declaration. 37 C.F.P. § 1.55(a) and 1.63. 
NOTE: This item is for any foreign priority for which the application being filed directly relates. If any parent 

U.S. application or International Application from which this application claims benefit under 35 U.S.C. 

§ 120 is itself entitled to priority from a prior foreign application, then complete item 18 on the ADDED 

PAGES FOR NEW APPLICATION TRANSMITTAL WHERE BENEFIT OF PRIOR U.S. APPLICATION® 

CLAIMED. 

10. Fee Calculation (37 C.F.R. § 1.16) 
A. 0 Regular application 



CLAIMS AS FILED 



Number filed Number Extra Rate Basic Fee 

37 C.F.R. § 1.16(a) 

$6§fr00 $710 .00 

Total 

Claims (37 C.F.R. 

§ 1.16(c)) 21 - 20 = 1 X $ 18.00 $18.00 

Independent 
Claims (37 C.F.R, 

§ 1.16(b)) 1 - 3 = 0 x $ 78.00 

Multiple dependent claim(s), 

if any (37 C.F.R. § 1.16(d)) + $260.00 



□ Amendment cancelling extra claims is enclosed. 

□ Amendment deleting multiple-dependencies is enclosed. 
I9£ Fee for extra claims is not being paid at this time. 

NOTE: If the fees for extra claims are not paid an filing they must be paid or the claims cancelled by amendment, 
prior to the expiration of the time period set for response by the Patent and Trademark Office in any 
notice of fee deficiency. 37 C.F.R. § 1.16(d). 

Filing Fee Calculation $ 728 - 0Q 

B. □ Design application 

($310.00—37 C.F.R. § 1.16(f)) 

Filing Fee Calculation $ 
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C. □ Plant application 

($480.00—37 C.F.R. § 1.16(g)) 

Filing fee calculation $ 

11. Small Entity Statement(s) 

□ Statement(s) that this is a filing by a small entity under 37 C.F.R. § 1 .9 and 1 .27 
is (are) attached. 

WARNING: "Status as a small entity must be specifically established in each application or patent in which 
the status is available and desired. Status as a small entity in one application or patent does not 
affect any other application or patent, including applications or patents which are directly or 
indirectly dependent upon the application or patent in which the status has been established. The 
refiling of an application under § 1.S3 as a continuation, division, or continuation-in-part (including 
a continued prosecution application under § 1.53(d)), or the filing of a reissue application requires 
a new determination as to continued entitlement to small entity status for the continuing or reissue 
application. A nonprovisional application claiming benefit under 35 U.S.C. § 119(e), 120, 121, or 
365(c) of a pnor application, or a reissue application may rely on a statement filed in the prior 
application or in the patent if the nonprovisional application or the reissue application includes a 
reference to the statement in the prior application or in the patent or includes a copy of the 
statement in the prior application or in the patent and status as a small entity is still proper and 
desired. The payment of the small entity basic statutory filing fee will be treated as such a reference 
for purposes of this section." 37 C.F.R. § 1.28(a)(2). 

WARNING: Small entity status must not be established when the person or persons signing the . . .statement 
can unequivocally make the required self-certification. " M.P.E.P., § 509.03, 6th ed., rev. 2, July 
1996 (emphasis added). 

(complete the following, if applicable) 

□ Status as a small entity was claimed in prior application 

/ , filed on , from which benefit 

is being claimed for this application under: 

35 U.S.C. § □ 119(e), 

□ 120, 

□ 121, 

□ 365(c), 

and which status as a small entity is still proper and desired. 
□ A copy of the statement in the prior application is included. 
Filing Fee Calculation (50% of A, B or C above) 

$ 

W07F: Any excess of the full fee paid will be refunded if small entitiy status is established and a refund request 
are filed within 2 months of the date of timely payment of a full fee. The two-month period is not 
extendable under § 1.13$. 37 C.F.R § 1.28(a). 

12. Request for International-Type Search (37 C.F.R. § 1.104(d)) 

(complete, if applicable) 



□ Please prepare an international-type search report for this application at the time 
when national examination on the merits takes place. 



13. Fee Payment Being Made at This Time 

E Not Enclosed 

□ No filing fee is to be paid at this time. 

(This and the surcharge required by 37 C.F.R. § 1.16(e) can be paid 
subsequently.) 

□ Enclosed 

□ Filing fee $ 

□ Recording assignment 
($40.00; 37 C.F.R. § 1.21(h)) 
(See attached "COVER SHEET FOR 
ASSIGNMENT ACCOMPANYING NEW 
APPLICATION".) 

□ Petition fee for filing by other than all the 
inventors or person on behalf of the inventor 
where inventor refused to sign or cannot be 
reached 

($130.00; 37 C.F.R. §§ 1.47 and 1.1 7(i)) 

□ For processing an application with a 
specification in 
a non-English language 
($130.00; 37 C.F.R. §§ 1.52(d) and 1.1 7(k)) 

□ Processing and retention fee 
($130.00; 37 C.F.R. §§ 1.53(d) and 1.21(1)) 

□ Fee for international-type search report 
($40.00; 37 C.F.R. § 1.21(e)) 

NOTE: 37 C.F.R. § 1.21(1) establishes a fee for processing and retaining any application that is abandoned for 
failing to complete the application pursuant to 37 C.F.R. § 1.53(f) and this, as well as the changes to 
37 C.F.R. §§ 1.53 and 1. 78(a)(1), indicate that in order to obtain the benefit of a pnor U.S. application, 
either the basic filing fee must be paid, or the processing and retention fee of § 1.21(1) must be paid, 
within 1 year from notification under § 53(f). 

Total fees enclosed $ 

14. Method of Payment of Fees 

□ Check in the amount of $ 

□ Charge Account No. in the amount of 

$ 

A duplicate of this transmittal is attached. 
NOTE: Fees should be itemized in such a manner that it is clear for which purpose the fees are paid. 37 C.F.R. 
§ 1.22(b). 



$ 

$ 

$ 
$ 
$ 
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15. Authorization to Charge Additional Fees 
WARNING: If no fees are to be paid on filing, the fallowing items should not be completed. 
WARNING: Accurately count claims, especially multiple dependent claims, to avoid unexpected high charges, 
if extra claim charges are authorized. 
□ The Commissioner is hereby authorized to charge the following additional fees 
by this paper and during the entire pendency of this application to Account No. 



□ 37 C.F.R. § 1.16(a), (f) or (g) (filing fees) 

□ 37 C.F.R. § 1.16(b), (c) and (d) (presentation of extra claims) 

NOTE: Because additional fees for excess or multiple dependent claims not paid on filing or on later presentation 
must only be paid or these claims cancelled by amendment prior to the expiration of the time period 
set for response by the PTO in any notice of fee deficiency (37 C.F.R. § 1.16(d)), it might be best not 
to authonze the PTO to charge additional claim fees, except possibly when dealing with amendments 
after final action. 

□ 37 C.F.R. § 1 .16(e) (surcharge for filing the basic filing fee and/or declaration 
on a date later than the filing date of the application) 

□ 37 C.F.R. § 1.17(a)(1H5) (extension fees pursuant to § 1.136(a)). 

□ 37 C.F.R. § 1.17 (application processing fees) 

NOTE: ". . .A written request may be submitted in an application that is an authorization to treat any concurrent 
or future reply, requiring a petition for an extension of time under this paragraph for its timely submission, 
as incorporating a petition for extension of time for the appropriate length of time. An authorization to 
charge all required fees, fees under § 1.17, or all required extension of time fees will be treated as a 
constructive petition for an extension of time in any concurrent or future reply requiring a petition for 
an extension of time under this paragraph for its timely submission. Submission of the fee set forth in 
§ 1.17(a) will also be treated as a constructive petition for an extension of time in any concurrent reply 
requiring a petition for an extension of time under this paragraph for its timely submission. " 37 C.F.R. 
§ 1.136(a)(3). 

□ 37 C.F.R. § 1.18 (issue fee at or before mailing of Notice of Allowance, 
pursuant to 37 C.F.R. § 1.311(b)) 

NOTE: Where an authorization to charge the issue fee to a deposit account has been filed before the mailing 
of a Notice of Allowance, the issue fee will be automatically charged to the deposit account at the time 
of mailing the notice of allowance. 37 C.F.R. § 1.311(b). 

NOTE: 37 C.F.R. § 1.28(b) requires "Notification of any change in status resulting in loss of entitlement to small 
entity status must be filed in the application . . . prior to paying, or at the time of paying, . . . the issue 
fee. . . " From the wording of 37 C.F.R. § 1.28(b), (a) notification of change of status must be made 
even if the fee is paid as "other than a small entity" and (b) no notification is required if the change 
is to another small entity. 
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16. Instructions as to Overpayment 

NOTE: ". ■ ■ Amounts of twenty-five dollar or less will not be returned unless specifically requested within 
a reasonable time, nor will the payer be notified of such amounts; amounts over twenty-five dollars may 
be returned by check or, if requested, by credit to a deposit account" 37 C.F.R. § 1.26(a). 

□ Credit Account No. 

□ Refund 
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Incorporation by reference of added pages 

(check the following item if the application in this transmittal claims the benefit of 
prior U.S. appiication(s) (including an international application entering the U.S. 
stage as a continuation, divisional or C-l-P application) and complete and attach 
the ADDED PAGES FOR NEW APPLICATION TRANSMITTAL WHERE BENEFIT OF 
PRIOR U.S. APPLICATIONS) CLAIMED) 

Q Plus Added Pages for New Application Transmittal Where Benefit of Prior U.S. 
Appiication(s) Claimed 

Number of pages added 5 



□ Plus Added Pages for Papers Referred to in Item 4 Above 

Number of pages added 



□ Plus added pages deleting names of inventor(s) named in prior application(s) 
who is/are no longer inventor(s) of the subject matter claimed in this application. 

Number of pages added . — - — 

□ Plus "Assignment Cover Letter Accompanying New Application" 

Number of pages added . 

Statement Where No Further Pages Added 

(if no further pages form a part of this Transmittal, then end this Transmittal with 
this page and check the following item) 

□ This transmittal ends with this page. 
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FORM 4-1. 



4-25 



Practitioner's Docket No. 



32803-0269659 



PATENT 



ADDED PAGES FOR APPLICATION TRANSMITTAL WHERE BENEFIT OF 
PRIOR U.S. APPLICATION(S) CLAIMED 

NOTE: See 37 C.F.R. § 1.78. 

17. Relate Back 

WARNING: if an application claims the benefit of the filing date of an earlier filed application under 35 U.S.C. 

§§ 120, 121 or 365(c), the 20-year term of that application will be based upon the filing date of 
the earliest U.S. application that the application makes reference to under 35 U.S.C. §§ 120, 121 
or 365(c). (35 U.S.C. § 154(a)(2) does not take into account, for the determination of the patent 
term, any application on which priority is daimed under 35 U.S.C. §§ 1 19. 365(a) or 365(b).) For 
a c-i-p application, applicant should review whether any claim in the patent that will issue is 
supported by an earlier application and, if not, the applicant should consider canceling the reference 
to the earlier filed application. The term of a patent is not based on a daim-by-daim approach. 
See Notice of April 14. 1995, 60 Fed. Reg. 20,195. at 20.205. 



□ Amend the specification by inserting, before the first line, the following sentence: 



NOTE: 'Any nonprovisional application daiming the benefit of one or more prior filed copending provisional 
applications must contain or be amended to contain in the first sentence of the specification following 
the title a reference to each such prior provisional application. Identifying it as a provisional application, 
and inducing the provisional application number (consisting of series code and serial number). "37 C.F.R. 



fiX "This application claims the benefit of U.S. Provisional Application® No(s).: 



(complete the following, If applicable) 



A. 35 U.S.C. § 119(e) 



§ 1.78(a)(4). 



APPUCATION NCKS).: 



RUNG DATE 



60 / 167,594 



11/26/99 



/. 
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B. 35 U.S.C. §§ 120, 121 and 365(c) 

NOTE: "Except for a continued prosecution application filed under § 1.53(d), any nonprovisional application 
claiming the benefit of one or more prior filed copending nonprovisional applications or international 
applications designating the United States of America must contain or be amended to contain in the 
first sentence of the specification following the title a reference to each such prior application, identifying 
it by application number (consisting of the series code and serial number) or international application 
number and international filing date and indicating the relationship of the applications. . . . Cross- 
references to other related applications may be made when appropriate." (See § 1.14(a)). 37 C.F.R. 
§ 1.78(a)(2). 

□ "This application is a 

□ continuation 

□ continuation-in-part 

□ divisional 

of copending application® 

□ application number 0 / — filed on " 

□ International Application _ filed on 

and which designated the U.S." 

The proper reference to a prior filed PCT application that entered the U.S. national phase is the U.S. 
serial number and the filing date of the PCT application that designated the U.S. 
(1) Where the application being transmitted adds subject matter to the International Application, then 
the filing can be as a continuation-in-part or (2) if it is desired to do so for other reasons then the filing 
can be as a continuation. 

The deadline for entering the national phase in the U.S. for an international application was clarified 
in the Notice of April 28. 1987 (1079 O.G. 32 to 46) as follows: 

"The Patent and Trademark Office considers the International application to be pending until the 22nd 
month from the priority date if the United States has been designated and no Demand for International 
Preliminary Examination has been filed prior to the expiration of the 19th month from the priority date 
and until the 32nd month from the priority date if a Demand for International Preliminary Examination 
which elected the United States of America has been filed prior to the expiration of the 19th month 
from the priority date, provided that a copy of the international application has been communicated 
to the Patent and Trademark Office within the 20 or 30 month period respectively. If a copy of the 
international application has not been communicated to the Patent and Trademark Office within the 
20 or 30 month period respectively, the international application becomes abandoned as to the United 
States 20 or 30 months from the priority date respectivley. These periods have been placed In the rules 
as paragraph (h) of § 1 .494 and paragraph § of § 1.435. A continuing application under 35 U.S.C. 365(c) 
and 120 may be filed anytime during the pendency of the international application. " 

□ "The nonprovisional application designated above, namely application 

/ __, filed , claims the benefit of 

U.S. Provisional Applications) No(s).: 



APPLICATION NO(S).: 



Where more than one reference is made above, please combine all references 
into one sentence. 
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18. Relate Back— 35 U.S.C. § 119 Priority Claim for Prior Application 

The prior U.S. applicatlon(s), Including any prior International Application designating the 
U.S., Identified above In Item 17B, in turn itself claim(s) foreign prlority(ies) as follows: 

Country Appln. no. Red on 

The certified copyfies) has (have) 

□ been filed on , In prior application 0 / , which was 



filed on _ 



□ is (are) attached. 

WARNING: The certified copy of the priority application that may have been communicated to the PTO by 
the International Bureau may not be relied on vAthout any need to file a certified copy of ^ priority 
application in the continuing application. This is so because the certified copy of the priority 
application communicated by the International Bureau is placed in a folder and Is not assigned 
a U. S. serial number unless the national stage is entered. Such folders are disposed of if the national 
stage is not entered Therefore, such certified copies may not be available if needed later In the 
prosecution of a continuing application. An alternative would be to physically remove the priority 
documents from the folders and transfer them to the continuing application. The resources required 
to request transfer, retrieve the folders, make suitable record notations, transfer the certified copies, 
enter and make a record of such copies in the Continuing Application are substantial. Accordingly, 
the priority documents In folders of international applications that have not entered the national 
stage may not be relied on. Notice of April 28, 1987 (1079 O.G. 32 to 46). 

19. Maintenance of Copandency of Prior Application 

NOTE: The PTO finds it useful if a copy of the petition filed in the prior application extending the term for 
response is filed with the papers constituting the filing of the continuation application. Notice of 
November 5, 1985 (1060 O.G. 27). 

A. □ Extension of time in prior application 

(This Item must be completed and the papers Wed In the prior application, 
If the period set in the prior application has run.) 

□ A petition, fee and response extends the term In the pending prior application 
until 

□ A copy; of. the petition filed in prior application is attached. 

B. Q; Conditional Petition for Extension of Time in Prior Application 

(complete this item, if previous item not applicable) 

□ A conditional petition for extension of time is being filed In the peming prior 
application. 

□ A copy of the conditional petition filed in the prior application is attached. 
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20. Further Inventorship Statement Where Benefit of Prior Application^) 
Claimed 

(complete applicable item (a), (ty and/or (c) below) 

(a) This application discloses and claims only subject matter disclosed in the prior 
application whose particulars are set out above and the inventors) in this 
application are 

□ the same. 

□ less than those named in the prior application, it is requested that the 
following inventors) identified for the prior application be deleted: 



(type name(s) of inventors) to be deleted) 
(b) E This application discloses and claims additional disclosure by amendment and 
a new declaration or oath is being filed. With respect to the prior application, 
the inventorfs) in this application are 
El the same. 

□ the following additional inventors) have been added: 



(type namefsj of inventorfs) to be added) 
(c) The inventorship for all the claims in this application are 
0 the same. 

□ not the same. An explanation, including the ownership of the various claims 
at the time the last claimed invention was made 

□ is submitted. 

□ will be submitted. 
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21 . Abandonment of Prior Application (if applicable) 

□ Please abandon the prior application at a time while the prior application is 
pending, or when the petition for extension of time or to revive in that application 
is granted, and when this application is granted a filing date, so as to make this 
application copending with said prior application. 

NOTE: According to the Notice of May 13, 1983 (103. TMOG 6-7), the filing of a continuation or continuation-in- 
part application is a proper response with respect to a petition for extension of time or a petition to 
revive and should include the express abandonment of the prior application conditioned upon the 
granting of the petition and the granting of a filing date to the continuing application. 

22. Petition for Suspension of Prosecution for the Time Necessary to 
File an Amendment 

WARNING: "The claims of a new application may be finally rejected in the first Office action in those situations 
where (A) the new application is a continuing application of, or a substitute for, an earlier application, 
and (B) all the claims of the new application (1) are drawn to the same invention claimed in the 
earlier application, and (2) would have been property finally rejected on the grounds of art of record 
in the next Office action if they had been entered in the earlier application. " M.P.E.P., § 706.07(b), 
7th ed. 

NOTE: Where it is possible that the claims on file will give rise to a first action final for this continuation application 
and for some reason an amendment cannot be filed promptly (e.g., experimental data is being gathered) 
it may be desirable to file a petition for suspension of prosecution for the time necessary. 

(check the next item, if applicable) 

□ There is provided herewith a Petition To Suspend Prosecution for the Time 
Necessary to File An Amendment (New Application Filed Concurrently) 

23. Small Entity (37 C.F.R. § 1.28(a)) 

□ Applicant has established small entity status by the filing of a statement in parent 
application / on . 

□ A copy of the statement previously filed is included. 
WARNING: See 37 C.F.R. § 1.28(a)- 

WARNING: 'Small entity statU3 must not be established when the person or persons signing the . . .statement 
can unequivocally make the required self-certification." M.P.E.P., § 509.03, 7th ed. (emphasis 
added). 

24. NOTIFICATION IN PARENT APPLICATION OF THIS FILING 

□ A notification of the filing of this 
(check one of the following) 

□ continuation 

□ continuation-in-part 

□ divisional 

is being filed in the parent application, from which this application claims priority under 35 
U.S.C. § 120. 
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Method and System for Rating Users, User-Contributed Items, 
Groupings of Related User-Contributed Items and Other 
Items 



Related Applications 

This application is related to and claims priority under 35 U.S.C. § 119(e) 
from U.S. Provisional Application No. 60/167,594 filed November 26, 1999, which is 
herein incorporated by reference. 

A. Technical Field 

This application relates to networks, such as computer networks and more 
specifically to a method and system for rating users, user-contributed items, groupings 
of related user-contributed items and other items on a network. 

B. Background 

A significant and distinctive feature of wide-area networks, including global 
wide area networks such as the Internet, is sometimes considered to be the ability of 
users anywhere on the network to access a centralized source for content in a 
particular category. The provision of information from centralized sources has a long 
tradition in print and electronic media. However, in many respects, despite the wider 
possibilities for dissemination, this is an "old media" concept, based on the economics 
of content production and information distribution before the advent of global wide 
area networks. Remarkably, this model still holds considerable sway, despite 
overwhelming changes in the technology and economics of information. More than 
ever before, any user anywhere has the ability not only to access centrally produced 
content, but also to interact with other users anywhere — at almost zero marginal cost. 

Nevertheless, several considerations have reduced the extent and growth of 
decentralized interactions over wide-area networks. It is difficult for a user to collect 
accurate information and form a meaningful opinion about the competence and 
credibility of another party without extensive contact — costly in time and effort. 
When interactions do occur, they are largely one-time or limited in number, with few 
expectations of repeat dealings. There is little incentive for users to share their 
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assessments of other users and such assessments cannot necessarily be taken at face 
value. 

Accessing raw, decentralized content that is time sensitive, without the ability 
to identify which users regularly produce items of higher Quality, means wading 

5 through an avalanche of biased, narrow or uninformed material obscuring valuable 
contributions. Even with the passage of time, after other users have reviewed the 
content and reacted to it, there may be more information to assess rather than less. 
User comments are rarely communicated in a form useable by others to filter content 
for Quality. By sheer volume, commentary is typically focused on controversial or 

1 0 negative content that has (often intentionally) prompted a strong reaction. 

Any system that rates Quality or content must achieve a critical mass of input 
in order to produce useful output. However, in the case of the Internet and other 
wide-area networks, ratings are systemically underprovided. The problem is 
particularly acute for content that is produced decentrally in a networked environment. 
15 When rating schemes seek input widely and make results freely available on public 
networks, users do not have appropriate incentives to contribute. 

Although users may engage in some rating, the possibility to "free ride" on the 
contributions of others discourages involvement. Users may respond to financial 
incentives to provide ratings, but it is difficult to monitor the Quality of these 
20 contributions. Also, if such incentives are financed through user fees restricting 
access, fewer users benefit from the ratings. 

Much of the prior art in the field of ratings has to do with predicting what is 
the best item of content to deliver next to a particular user — often for strictly 
commercial purposes — based on the user's revealed preferences and current position 
25 on the network, so-called "collaborative filtering." The mechanism of this approach is 
generally to associate each user with other users of like mind who have been given 
appropriate incentives to provide ratings, or who have otherwise provided ratings 
incidentally or voluntarily. 
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Such a method does little, however, to assist users exploring new subject 
matter. The emphasis is not absolute item Quality, but the relative likes and dislikes 
of different people and the objective is to provide a user who has revealed certain 
preferences with more of the same. At best, such a method determines how the user, 
5 with the user's current breadth of knowledge and experience, might rate items. It does 
little to identify experts acknowledged by other people who have themselves 
demonstrated talent. 

The inadequacies of conventional rating systems have exacerbated the 
centralized nature of content production and filtering on the Internet. Groups of 
10 online "experts," such as the staff or affiliates of Internet portals or community web 
sites with tight, centralized editorial control, play an excessive role. As a result, fewer 
and less meaningful interactions occur that take advantage of the rich, decentralized 
resources of wide-area networks. 

Summary of Preferred Embodiments 

15 The described embodiments of the present invention offer alternative 

structures for decentralized interactions among users on wide-area networks and a 
method of constructing, applying and distributing ratings of users, user-contributed 
items, groupings of related user-contributed items and other items. 

In certain of the described embodiments, the higher the Quality of the items 
20 contributed by or associated with a particular user, the more weight assigned to ratings 
supplied by such user. At the same time, a calculation using weighted ratings 
determines item Quality. In effect, the weights assigned to ratings affect Quality and 
Quality affects the weights assigned to ratings, simultaneously. The preferred 
embodiments include a solution to the "circular" character of this approach. 

25 The described embodiments include a series of mathematical methods 

embedded in network processes. These network processes provide for a series of 
interactions between user nodes, mediated by other network elements that create a 
structured environment. Among other things, user interactions can include 
participation in a web site's online discussion group or chat facilities, an e-mail based 
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mailing list, unidirectional, bidirectional or widely broadcast digital video or audio 
communication, a distributed communication facility such as Usenet newsgroups or 
IRC chat, or an online auction or other facility through which users provide, or agree 
to provide, goods and services in commerce, in each case enhanced with features of 
5 the described embodiments. 

Among such enhancements are facilities that provide participants with an 
opportunity to rate each instance of one user interacting with other users, referred to 
herein as an "item." For example, a chat session would generally be considered as a 
separate item for each user who participated, for example by injecting at least one 
10 statement into the session during a particular hour. As another example, particular 
charities might be considered to be items. Users can post their ratings of the charities, 
and the charities are assigned a Quality value as described below. 

In certain preferred embodiments, ratings of items contributed by or associated 
with a particular user are used to construct measurements of the user's competence 
15 and credibility as a participant in a networked environment, called herein "Expertise" 
and "Regard." In general, the term "Expertise" is used in connection with the 
"Expert" methods described herein and the term "Regard" is used in connection with 
the "High Regard" methods described herein. 

Either Expertise or Regard can be used as a factor in determining the relative 
20 weight assigned to each user's evaluations of items contributed by or associated with 
other users, producing a measure of the relevance, accuracy and importance of a 
particular item, referred to herein as item "Quality." For example, the Expertise or 
Regard of users rating a particular discussion group posting might be used to weight 
their ratings in a calculation of the posting' s Quality. 

25 Either Expertise or Regard may also be used to predict the likely relevance, 

accuracy and importance of a user's contributed items (both historical and future), for 
example, when a particular item is too new to have itself received ratings or when 
ratings of a particular item are sparse. 
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Either Expertise or Regard may also serve as an independent benchmark 
relevant to an evaluation of the user for other purposes, including purposes not 
associated with interactions across wide-area networks. 

Expertise, Regard or Quality, or related measures, may be bundled with 
5 relevant content and transmitted to nodes on a network, offering multiple additional 
opportunities to enhance user interactions, in some embodiments. One or more of 
these measures may also be used to filter, highlight, sort or otherwise evaluate items 
or to limit a user's interactions to items and other users who meet minimum standards. 

Items contributed by users who regularly contribute items of poor Quality or 
10 interject negative, disruptive or misleading items into the system through 
incompetence or ill will, can be identified, downgraded or even removed from view. 
Equally, items by users who regularly provide items of high Quality can be given 
immediate prominence, even before receiving a single user rating. 

High ratings can raise the profile of any item, irrespective of the Expertise or 
15 Regard of the user who contributed it. Also, an item that misses the mark and receive 
poor ratings can be identified, downgraded or removed from view, even if the user 
who contributed it has high Expertise or Regard. 

Arguably, these methods benefit not only users in general, but also the 
contributors, who would want more visibility for their best items. 

20 The described embodiments of applicants' invention also include methods 

enabling users to "Vouch" for items contributed by another user by associating their 
own Expertise or Regard with such items. In this manner, a well Regarded user (also 
called a highly Regarded user) can quickly raise the prominence of a valuable item 
contributed by an unknown or otherwise poorly Regarded user. Similarly, users are 

25 enabled to "Discredit" items contributed by other users by asserting their own 
Expertise or Regard in opposition to such items. In these cases, the Expertise or 
Regard not only of the contributor, but also of the user who Vouched for or 
Discredited such items can be enhanced or diminished depending on other user's 
ratings of such items. 
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To keep ratings current and relevant, there are also methods to remove ratings 
from the calculation of Expertise or Regard over time, based on the age of items, the 
Expertise or Regard of the user and other factors. 

To facilitate the accumulation of a critical mass of data, certain embodiments 
5 will construct an environment in which users provide ratings as a natural part of 
navigating through items. For example, if the user views an item for a period of time 
suggesting the user has given it attention and consideration, the user may be 
encouraged or required either to provide a rating or to exit the system, or the portion 
of the system the user is currently interacting with. Also, for example: the act of 
10 moving from one item to another at a particular moment, when certain visual cues 
(among a rotating series of alternatives) appear on the user interface, may indicate that 
the user has selected a particular rating. The intention is to create an efficient manner 
for users to provide ratings, to limit the requirement to provide ratings to situations in 
which the user has actually reviewed the item and to generate a large body of useful 
15 rating data. By providing an efficient user interface and striking an appropriate 
balance, the breadth, Quality and utility of systematic ratings- more than compensates 
users for the additional effort required to supply ratings. 

Advantages of the described embodiments will be set forth in part in the 
description which follows and in part will be obvious from the description, or may be 
20 learned by practice of the described embodiments. The objects and advantages of the 
described embodiments will be realized and attained by means of the elements and 
combinations particularly pointed out in the appended claims and equivalents. 

Brief Description of the Drawings 

Fig. 1 shows a network, various users of the network and a network server. 

25 Fig. 2(a) shows an example of a user interface allowing users to view and rate 

items. 

Fig. 2(b) describes alternative rating schemes that can be used in accordance 
with the present invention. 



60195430vl 



6 



Fig. 2(c) shows a web page incorporating a rating scheme in accordance with 
the present invention. 

Fig. 3 shows a simple example in a system having only four users 
demonstrating that ratings supplied by users are not necessarily valued equally. 

5 Fig. 4(a) and 4(b) show a simple example demonstrating that a user's Regard 

is affected by how other users rate the items that the user contributes and by the 
Regard of those users, and that one user's Regard can affect the Regard of other users 
and the Quality of items contributed by other users. 

Figs. 5(a) and 5(b) show another simple example demonstrating that a user's 
10 Regard is affected by how other users rate the items that the user contributes and by 
the Regard of those users, and that a user's Regard can affect the Regard of other 
users and the Quality of items contributed by other users. 

Figs. 6(a) and 6(b) show a simple example demonstrating that a user's 
Expertise is affected by how others rate the items that the user contributes and that a 
1 5 user's Expertise can affect the Quality of items contributed by other users. 

Fig. 7(a) is an overview of data flows in an example High Regard ratings 
system, showing the circular nature of Regard affecting the Regard of other users and 
the Quality of items contributed by other users. 

Fig. 7(b) is a more detailed overview of data flows in an example High Regard 
20 ratings system, showing the circular nature of Regard affecting the Regard of other 
users and the Quality of items contributed by other users, emphasizing procedures for 
the flow, storage and interpretation of data. . 

Figs. 7(c)-7(e) are flow charts showing the inputs and outputs to "basic" 
determinations of item Quality, user Regard and user Expertise. 

25 Fig. 8 is a plot of a Regard Inertial Model. 

Fig. 9(a) is a plot of a Quality Inertia Model. 

Fig. 9(b) is a plot of a Segmented Decay Model. 
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Fig. 10 shows an example of a small Regard data set. 

Fig. 1 1 shows an example of a data structure used to store and retrieve data 
required to perform calculations of Regard and Quality in a preferred embodiment of 
the present invention. 

Fig. 12(a) is a block diagram of a first example forum server application 
including a rating server. 

Fig. 12(b) is a block diagram of another example forum server application 
including a rating server, where the rating information is sent directly to a user's web 
page. 

Fig. 12(c) is an example of a web page generated in accordance with the block 
diagram of Fig. 12(b). 

Fig. 13 is a block diagram of another example forum server application 
communicating with a separate communication forum. 

Fig. 14 is a block diagram of another example forum server application 
communicating with a separate communication forum and an ad server. 

Fig. 15(a) is a flow diagram showing communication between elements of Fig. 
14 during forum/thread index intercommunication. 

Fig. 15(b) is a flow diagram showing communication between elements of Fig. 
14 during article view intercommunication. 

Fig. 15(c) is a flow diagram showing communication between elements of Fig. 
14 during post/reply intercommunication. 

Figs. 16(a)- 16(h) are flow charts showing methods used during the 
intercommunication processes of Figs. 15(a) - 15(c). 

Fig. 17 is a block diagram of an example e-commerce server. 

Fig. 18 is a block diagram of an example auction server and a rating server. 
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Fig. 19 is a block diagram of an example server for an individual and 
commercial rating service. 

Figs. 20(a) and 20(b) are, respectively, a graph and a table that aid in 
explaining the concepts of "Vouching" and "Discrediting." 

5 Detailed Description of Embodiments 

1. Overview 

Fig. 1 shows a network 100, such as the Internet, an intranet, a wide-area 
network (WAN), a wireless or telephonic network, or any other appropriate network. 
Network 100 can also be a combination of various types of networks or of various 

10 networks and sub-networks. Users communicate via the network 100 by sending 
information by way of methods and protocols appropriate to the network 100. 
Although it will be understood that many users can access network 100 
simultaneously, three users 110, 120 and 130 are shown. A user can be a human 
being or another entity, such as a computer program capable of accessing network 

15 100. Fig. 1 also shows a rating server 140 as discussed below. 

In the network of Fig. 1, users contribute items, such as, for example, e-mail 
messages or discussion group postings. (As discussed below in a separate section, the 
term "items" encompass a wide variety of things.) The users view each other's items 
and some or all of the users rate the items contributed by other users. For example, in 

20 Fig. 1, user 110 reads other items and rates them, but does not contribute items 
himself. In Fig. 1, user 120 reads others' items but does not rate them. User 120 
does, however, contribute items. In Fig. 1, user 130 reads others' items and rates 
them. User 130 also contributes items. The three users discussed herein are intended 
to show that various users interact with network 100 in different ways. 

25 In Fig. 1, user 1 10 receives items 1 12 contributed by other users, including the 

Quality of the items and the Regard of the users who contributed the items. It should 
be understood that, although Fig. 1 discusses a system using the Regard of users, Fig. 
1 could also apply to a system that uses the Expertise of users (as both terms are 
defined herein). Only Regard is shown in the example of Fig. 1 to aid in maintaining 

30 the simplicity of the example. User 1 10 contributes a rating 1 14 of one or more of the 
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items. User 120 receives items 122 contributed by other users, including the Quality 
of the items and the Regard of the users who contributed the items. User 120 
contributes one or more items 126 of his own. User 130 receives items 132 
contributed by other users, including the Quality of the items and a Regard of the 
users who contributed the items. User 130 contributes a rating 134 of one of more of 
the items. User 130 contributes one of more items 136. 

It will be understood that Fig. 1 is provided by way or example and not 
limitation. The invention can be implemented in a network as shown, or in other 
environments, such as, for example, a database in which users enter, read and access 
items in the database. The invention could also be implemented in, for example, an e- 
mail system operating in a networked environment. It will be understood that, in the 
described embodiments, the functionality described herein is preferably performed by 
a data processing system or systems performing instructions stored on a medium or 
memory accessible by the data processing system(s). The invention is not limited to 
the architecture, programming models, protocols or procedures shown and described 
herein. 

Fig. 2(a) shows an example of a user interface allowing users to view and rate 
items. Although not shown, users can also contribute their own items. Fig. 2(a) 
shows a web page that displays messages in an online discussion forum. The text of a 
current message is displayed in an area 202. As the user views a current message, he 
can rate the message using rating area 204 (or a similar rating scheme, such as one of 
the alternative ratings schemes described below). 

The user interface of Fig. 2(a) uses alternating visual cues. Specifically, in the 
ratings scheme shown in the Figure, five diamond boxes 204 are displayed. One of 
these diamonds is highlighted per second, with the highlights preferably moving 
sequentially from left to right and wrapping from diamond #5 back to diamond #1. 
The pattern repeats until a predefined user action is detected. A predefined user action 
can be, but is not limited to a keystroke, mouse movement or click, voice command 
and/or another appropriate command. When a predefined user action is detected, it 
selects a rating corresponding to a currently highlighted diamond in area 204. In 
certain embodiments, such a predefined user action also causes the user to proceed to 
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the next item, thread, or elsewhere. For example, the user clicks on a rating and 
automatically advances to a next item, thread, etc. 

The rating diamonds preferably correspond to varying ratings, from, for 
example, lowest to highest ratings. Thus, the user merely needs to wait until a rating 
that he agrees with is highlighted and perform one of the predefined user actions, such 
as slightly moving his mouse or touching any key on the keyboard. The user does not 
have to move a mouse controlled pointer to a particular location on the web page, or 
click or highlight specific objects or locations on the web page to rate messages, since 
the act of rating automatically causes the display to advance to the next message. 

In a preferred embodiment, the interface of Fig. 2(a), including diamonds 204, 
is implemented via an applet on the web page. The interface shown could be 
implemented using any appropriate method, such as via a browser plug-in, Java code 
transmitted during a particular network connection and persisting for a limited period 
of time, or a standard html web page. 

Fig. 2(b) describes alternative rating schemes that can be used in accordance 
with the present invention. While Fig. 2(b) describes the "single click rating" scheme 
of Fig. 2(a), it also describes several other rating schemes. These include a "rate 
while you navigate" scheme and a "single keystroke/click rate/Navigate" scheme. In 
the rate while you navigate scheme, use of keyboard or mouse commands to navigate 
to other items while the visual cues are alternating is interpreted as selecting the rating 
represented by the currently highlighted visual cue ("rate while you navigate"). In the 
single keystroke/click rate/navigate scheme, the rating diamonds 242 are not 
automatically highlighted. Instead, the user clicks on an appropriate diamond or 
presses one of keys 1, 2, 3 or 4 on the keypad (or some other predetermined keys) to 
both select a rating for the current message and to go to the next article, next thread, 
out or elsewhere. 

Although not shown, other ratings schemes do not automatically advance the 
user to a next view, but require the user to perform the rating and navigation functions 
separately (even though one or the other can still be accomplished by alternating 
indicators as discussed above). For example, an alternative rating scheme places the 
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ratings diamonds next to each message or each thread visible on the page. Still 
another scheme places the rating diamonds next to the messages. 

A message "thread" is a group of related messages having a linear order and a 
hierarchy of levels of indentation (representing interrelationship) so that a user can 
view the next or previous item in the message thread. It is important to note that, in 
the described embodiment, it is the messages, message threads, or other items that are 
being rated, not the authors/contributors of the items. Still another scheme allows the 
user to type a rating into a box on the web page or enter a rating in a drop-down menu 
or special window. Still another rating scheme allows the user to move a slider bar or 
similar non-discrete input device on a web page and translates the user's action into a 
numeric rating. Thus, the above user interfaces could also be implemented as non- 
discrete interfaces wherein user ratings fall along a spectrum (for example, between 0 
and 1) and are not limited to predetermined discrete values. 

Fig. 2(c) shows a web page incorporating a rating scheme in accordance with 
the present invention. In this example, the ratings diamonds invite the user to rate the 
entire page, not just the contents of certain areas on the page. The rating diamonds 
can alternate as shown above or require the user to click on or enter an appropriate 
rating. 

Still another rating scheme rates products and services offered on and 
transactions completed through a networked commerce facility, such as an online 
auction service. In a transaction-rating scheme, the item can be, for example, a 
completed transaction in which the user was the buyer or seller or another user 
monitoring the quality of the participation by the buyer or seller, or of the goods or 
services that are the subject of the transaction. Other types of items can also be rated 
in such a system. In such a rating scheme, Quality will attach to various transactions 
from a buyer's perspective and a seller's perspective. For example the buyer may rate 
the seller highly, but the seller may rate the buyer much lower. Thus, the same 
transaction receives very different ratings from buyer and seller, both of whom are 
"the author" of the transaction in some sense. Alternatively, the buyer behavior and 
the seller behavior can be considered two separate items that are rated separately. In 
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general, in a product and services ratings scheme, any user may be able to provide a 
rating or only a limited number of users may provide ratings. 

Fig. 2(a) also shows a Regard diamond 206 (which represents the Regard of 
the author of the current item) and a Quality diamond 208 (which represents the 
Quality value assigned to the current item). In general, Regard can be used alone as 
an independent benchmark, as a predictor of the value of the user's historical and 
future contributions or, in a system-wide calculation, as a factor determining the 
relative weight assigned to the user's evaluations of other user's contributions. The 
Quality of an item is based on the ratings assigned to the item by other users and also 
based on the Regards of those other users. A discussion of the meaning of and 
preferred methods of obtaining Regard and Quality are discussed below in detail. It 
should be understood that an interface similar to that of Fig. 2(a) could also be used to 
display item Quality and user Expertise in a system that uses Expertise values rather 
than Regard values. 

Fig. 2(c) also shows a ranking diamond 244. This diamond represents the 
Quality of the web page. In this example, no web page author has been identified. 
This example demonstrates that Quality can be displayed for an item even though the 
item is not associated with a user. Alternately, both Regard (of the author) and 
Quality (of the page) could be displayed for the web page, in a manner similar to that 
shown in Fig. 2(a). Expertise could be used in place of Regard. 

Figs. 7(c)-7(e) are flow charts showing the inputs and outputs to "basic" 
determinations of item Quality, user Regard and user Expertise. In certain 
embodiments, each user is assigned a Regard value, in accordance with ratings that 
other users assign to items contributed by that user and in further accordance with the 
Regard values of the users who contribute the ratings. In certain other embodiments, 
each user is assigned an Expertise value, in accordance with ratings that other users 
assign to items contributed by that user. In certain preferred embodiments, a user is 
assigned a default Expertise or Quality value. Quality values are determined for each 
item contributed by a user, whether the systems implements Regard or Expertise. The 
Quality value of an item is determined in accordance with the ratings received for the 
item and the Regard (or Expertise) of the users who contributed the ratings. Details of 
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the determination of "basic" Quality, Regard and Expertise are discussed below in 
Section 5 et seq. Variations on the basic determinations are also discussed below in 
Section 5 et seq. 

Figs. 3-6 show a simple example in a system having only four users 
demonstrating that ratings supplied by users are not necessarily valued equally in a 
system using Regard. In the following examples, Figs. 3-5 show a system 
implementing Regard values. Fig. 6 shows a system implementing Expertise values. 
To enhance the clarity of the examples in Figs. 3-6, it is assumed that the system has 
only four users and that only one item is contributed and rated at a time. It is 
contemplated that most systems employing an embodiment of the present invention 
will have many more users and that multiple item contributions and multiple ratings 
by the users of other user's items will overlap in time. 

In Fig. 3, User 4 contributes Item A, which is viewed and rated by Users 1, 2 
and 3. In the example, User 1 is well Regarded (having, e.g., a Regard of 0.9). User 2 
is poorly Regarded, having a Regard of 0.1. User 3 is also poorly Regarded, having a 
Regard of 0.1. (Each User's Regard is based at least in part on his previous 
contributions and the ratings of other users for those contributions, as discussed below 
in detail). 

This example assumes that possible ratings fall between 0 and 1 inclusive. 
Other systems may, of course, use other ratings scales. In Fig. 3, well Regarded User 
1 "loves" Item A, giving it a rating of 1.0 (the highest rating possible). Poorly 
Regarded User 2 "hates" Item A, giving it a rating of 0.0 (the lowest rating possible). 
Poorly Regarded User 3 also "hates" Item A, giving it a rating of 0.0. Even though 
two users hate Item A, both of those users are poorly Regarded. Thus, Item A 
receives a Quality value that is closer to the rating by well Regarded User 1 than to 
the ratings received from User's 2 and 3. Thus, in a system using Regard, the ratings 
contributed by all users are not equally valued. Ratings from well Regarded users are 
valued more when determining the Quality of an item. It should be noted that the 
terms "love" and "hate" are presented here as an aid to understanding. The 
embodiment described here determines numeric ratings. For example, a user interface 
may allow a user to choose "love" or "hate" or some other word suggesting a reaction 



601 95430v 1 



14 



somewhere in between, but these non-numeric choices are eventually translated into 
numeric values for the purpose of calculations in the preferred embodiments. 

Figs. 4(a) and 4(b) show a simple example in a system having only four users 
demonstrating that a user's Regard is affected by how others rate the items that the 
5 user contributes and by the Regard of those users and that a user's Regard can affect 
the Regard of other users and the Quality of items contributed by other users. Again, 
for the purposes of this simple example, it is assumed that only one item is 
contributed and rated at a time, although this would not necessarily be the case in a 
real situation. 

10 Fig. 4(a) shows that, once users have rated Item A in Fig. 3, the ratings for 

Item A affect the Regard of User 4, who contributed Item A. In the example, the new 
item contributed by User 4 receive high ratings by well Regarded users and User 4's 
Regard value goes up. In the example, User 4's Regard value rises from 0.5 (middle 
of the scale used here) to some higher value. 

15 Fig. 4(b) continues the example of Fig. 4(a), showing that User 4's changed 

Regard value affects the Regard and the item Quality of other users whose items User 
4 has rated in the past. In the example, User 4 has historically given low ratings to 
items contributed by User 3. When User 4's Regard rises, it negatively affects the 
Quality value for items contributed by User 3 that were previously rated by User 4. 

20 The rise in User 4's Regard also negatively affects the Regard of User 3 (not shown), 
since User 3's Regard is based on the ratings his items received from other users and 
on the Regard values of those users. In the example, User 4 has historically given 
high ratings to items contributed by User 1. When User 4's Regard rises, it positively 
affects the Quality value for items contributed by User 1 that were previously rated by 

25 User 4. The rise in User 4's Regard also positively affects the Regard of User 1 (not 
shown), since User l's Regard is based on the ratings his items received from other 
users and the Regard values of those users. The example of Fig. 4(b) could be 
expanded to show more steps in which the newly affected Regard of Users 1 and 3 
affect the Regard of other users whose items Users 1 and 3 have rated, and the Quality 

30 of those items, and so on. 
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New ratings for an item of one user can cause a change in the Quality of that 
item. Similarly, new ratings for an item of one user can cause a change in the Regard 
of that user, which can potentially cause a change in the Regard of all users and in the 
Quality of items for all users. 

Figs. 5(a) and 5(b) show another simple example in a system having only four 
users showing that a user's Regard is affected by how others rate the items that the 
user contributes and by the Regard of those users and that a user's Regard can affect 
the Regard of other users and the Quality of items contributed by those other users. 

Fig. 5(a) shows that a user's Regard can be adversely affected as well. Again, 
it is assumed that the system has only four users and that only one item is contributed 
and rated at a time. In Fig. 5(a), User 1 contributes Item B, which is viewed and rated 
by Users 2, 3 and 4. In the example, User 1 was well Regarded when he contributed 
Item B. User 2 is poorly Regarded, having a Regard of 0.1. User 3 is also poorly 
Regarded, having a Regard of 0.1. User 4 is well Regarded, having a Regard of 0.9. 
In Fig. 5(a), poorly Regarded User 3 "loves" Item B, giving it a rating of 1 .0 (which in 
our example is the highest possible rating). Similarly, poorly Regarded User 2 
"loves" Item B, giving it a rating of 1.0. In contrast, well Regarded User 4 "hates" 
Item B, giving it a rating of 0.0. Even though two users love Item B, both of those 
users are poorly Regarded. 

Fig. 5(a) shows that, once the users have rated Item B, the rating for Item B 
adversely affects the Regard of User 1, who contributed Item B. In the example, the 
new item contributed by User 1 receives low ratings from well Regarded users (and 
below the average ratings for User l's previous items). Therefore, in the example, 
User l's Regard value falls to 0.5 (in the middle of the range) from some higher value. 

Fig. 5(b) continues the example of Fig. 5(a), showing that User l's changed 
Regard value affects the Regard of other users and the Quality of items contributed by 
other users whose items User 1 has rated in the past. In the example, User 1 has 
historically given low ratings to items contributed by User 2. When User l's Regard 
falls, it positively affects the Quality value for items contributed by User 2 that were 
previously rated by User 1, because his low ratings of these items are given less 
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weight. The fall in User l's Regard also positively affects the Regard of User 2, since 
User 2's Regard is based on the ratings his items received from other users and the 
Regard values of those users. In the example, User 1 has historically given high 
ratings to items contributed by User 4. When User l's Regard falls, it negatively 
5 affects the Quality value for items of User 4 that were previously rated by User 1, 
because his high ratings of these items are given less weight. The fall in User l's 
Regard also negatively affects the Regard of User 4, since User 4's Regard is based on 
the ratings his items received from other users and the Regard values of those users. 
The example of Fig. 5(b) could be expanded to show more steps in which the newly 
10 affected Regard of Users 2 and 4 affect the Regard of other users whose items Users 2 
and 4 have rated, and the Quality of those items, and so on. 

The examples of Figs. 4 and 5 have involved systems that implement a Regard 
value for users. In contrast, Figs. 6(a) and 6(b) show a simple example in a system 
that implements an Expertise value for users. The example shows that a user's 
1 5 Expertise is affected by how others rate the items that the user contributes and that a 
user's Expertise can affect the Quality of others' items. Note however, that a change 
in the Expertise value of a user does not affect the Expertise values of other users. A 
user's Expertise value is changed when other users submit new ratings for that user's 
items. 

20 Fig. 6(a) shows an example in which User 4 contributes Item C, which is rated 

by Users 1, 2 and 3. User 2 has a high Expertise, but "hates" the item and give it a 
low rating. Users 1 and 3 both have low Expertise values, but "love" the item and 
give it a high rating. User 4's Expertise is based on the arithmetic mean of all ratings 
for all items contributed by User 4. Here, the new high ratings from Users 1 and 3 

25 have raised the mean of the ratings of User 4's items. Thus, the ratings from other 
users who rate User 4's items affect user 4's Expertise, but the Expertise of those 
other users does not affect User 4's Expertise. It is irrelevant that the users who love 
Item C both have low Expertise values. It should be noted that, in a system 
implementing Expertise instead of Regard, Quality is determined in much the same 

30 way as shown in Fig. 3 and as described below. That is, the basic Quality 
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determination for an item is made in accordance with ratings received for the item, 
weighted by Expertise values of the users providing the ratings. 

Fig. 6(b) continues the example of Fig. 6(a), showing that User 4's changed 
Expertise value affects the item Quality of other users whose items User 4 has rated in 
5 the past. In the example, User 4 has historically given low ratings to items 
contributed by User 1 . When User 4's Expertise rises, it negatively affects the Quality 
value for items of User 1 that were previously rated by User 4. The rise in User 4's 
Expertise does not affect the Expertise of User 1. In the example, User 4 has 
historically given high ratings to items contributed by User 2. When User 4's 
10 Expertise rises, it positively affects the Quality value for items of User 2 that were 
previously rated by User 4. The rise in User 4's Expertise does not affect the 
Expertise of User 2. 

New ratings for an item of one user can cause a change in the Quality of that 
item. Similarly, new ratings for an item of one user can cause a change in the 
15 Expertise of that user, which can potentially cause a change in the Quality of items for 
all users. 

Again, it is emphasized that the above examples are simple examples, 
provided to aid in a basic understanding of the described embodiments of the present 
invention. A more precise discussion is provided below. The following paragraphs 
20 provide a discussion of terms and terminology used herein. 

2. Items 

The growth of the Internet and other wide area networks has multiplied the 
forms of and opportunities for, decentralized interactions. An item may be composed 
25 of words, whether ASCII text or text formatted by a word processing technology or 
hypertext mark-up language, text contained in a collection of data packets constituting 
an e-mail message, discussion group posting or Usenet newsgroup article, or the 
output of a process that translates from one language to another, or written words into 
spoken words, or spoken words into written words. An item may also be an 
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interactive, sequential exchange of words making up a group or one-to-one chat 
session. 

An item may be a fixed visual image, whether a drawing or image captured by 
a digital camera, or transferred from a photographic original to a scanned 
5 representation of the original. Streaming audio or video, whether live or previously 
recorded material and whether unidirectional, bidirectional or broadcast, would 
constitute an item, as would words, images, or streaming audio or video integrated 
into a web site. 

More broadly, any single behavior or collection of behaviors viewable by 
1 0 others, whether online or offline, may constitute an item. An example of this includes 
the course of a user's conduct in an online auction, whether as buyer or as seller. 
Another example is the performance offline of a subcontractor or of a general 
contractor, in their respective professional roles — whether assessed by each other or 
a third party — who entered into their agreement for the performance of services 
1 5 using an online medium. 

An item may also be information, goods and services, or assets which one user 
recommends to other users, or otherwise associates with oneself or one's reputation. 
An example of this is a recommended link to a third party website, or a link deep into 
the structured hierarchy of a website. Another example is an asset one puts forth for 
20 sale in an online auction. Another example is software, whether distributed as source 
code or in executable form and whether constituting a stand-alone program or 
operating system, or a replacement of or addendum to a portion thereof. An item may 
exist only as a pointer in records to locations accessible over, or data streams 
transmitted across, a wide area network. 

25 3. Users 

In various embodiments of the present invention, different users or types of 
users may have items associated with them and different users or types of users may 
participate in the calculation of Expertise, Regard, Quality and other measures 
included in the described embodiments of the present invention. 
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In certain embodiments, users will overtly choose to participate. For example, 
a user might access a mailing list, discussion group, chat session or other items or 
grouping of items via a facility, such as a website, which is specifically enabled with 
features of the described embodiments of the invention. In some embodiments, this 
5 facility will be the only avenue to access the items or grouping of items and users 
would anticipate receiving in the ordinary course information regarding the Regard 
and Expertise (and related measures) of users and regarding the Quality (and related 
measures) of items. 

In other embodiments, some users need not ever access such a facility, or give 

10 any indication of interest in, or the intention of, assigning ratings to items contributed 
by other users, or give direct consent to the application of the measurements and 
methods of the described embodiments to themselves or to items they contribute. For 
example, a user might participate in a mailing list, discussion group, chat session or 
other grouping of items that is accessed by multiple facilities, only a subset of which 

15 are enabled with features of the described embodiments of the invention. More 
specifically, a user might choose to read and post Usenet newsgroup messages via a 
desktop application or website that has no support for features of the described 
embodiments, while other users access or contribute such postings via a facility that 
supports features of the described embodiments. 

20 In such embodiments, users who contribute items via facilities that are not 

enabled with any or all features of the described embodiments may also be 
incorporated into databases as item contributors and assigned Expertise or Regard 
values. The items contributed by such users may therefore be assigned Quality or 
related measurements and be subject to Vouching and Discrediting by other users and 

25 the other methods of the described embodiments, as discussed in various section 
below. 



4. Notation 

The following section discusses a preferred method of determining Quality 
30 and Regard from ratings supplied by users. User Expertise is also discussed. 
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Let p be the number of participating users, p grows to include each user added 
over time. 



Let i and j represent two users, where 

i,j € {1, . . . ,p} 

Let c/ be the number of items contributed by user i. 



4.1 Ratings 



if user i = user j 
) otherwise 



where n is an item contributed by user i, with 

ne {!,■■■ ,a} 

And v Z is behavior by user j in response to item n, which behavior is mapped by the 
function H x ) to a floating point variable 9i i , with. 

0<sS<l. 

In alternative embodiments, 5 S can be any numerical scale for ratings, or any 
scheme that orders items to reflect user assessments or preferences. 

In alternative embodiments, behavior u 5 may include overt selection of a 
rating with a mouse or keyboard interface, a voice command, a movement of the hand 
or eye or other observable behavior by the human body. However, it is not necessary 
that ratings be overt, or that the user even be aware of participating in a ratings 
system. For example, behavior can also include the time spent viewing an item, the 
number of comments one interjects into a chat session, the decision to click on an ad 
banner, or a decision to bid on an auction opportunity. In the case of an item 
organized into a hierarchy, such as a web page with multiple levels of depth or 
specificity under subject categories, the behavior that triggers a rating may be the 
number of levels one traverses to lower levels. 
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In the event that user j does not connect with, engage in, or be associated with 
any behavior related to, the item, then 9i i = 0 . 

In certain preferred embodiments, such as those shown in Figs. 2(a)-2(c), the 
user interface can present sequentially a repeating series of highlighted or colored 
5 diamonds 242, or other alternating visual cues, each associated with a particular value 
of the rating variable. In a preferred embodiment, the rate at which the visual cues 
242 alternate is one second, although other intervals can be used. Upon any use of the 
keyboard or mouse, the currently colored or highlighted visual cue is interpreted as 
the user's rating ("single-click rating"). In order to limit ratings to items that a user 

10 has had a meaningful opportunity to review, in certain embodiments the visual cues 
and rating opportunity are not presented to the user until after the user has navigated 
to the end of the item and a fixed number of seconds has elapsed since delivery of, or 
the beginning of the user's interaction with, the item. In a preferred embodiment, the 
interval is 10 seconds. In certain other embodiments, use of keyboard or mouse 

15 commands to navigate to other items is interpreted as selecting the rating represented 
by the currently highlighted visual cue ("rate while you navigate"). Expressed 
differently, rating is an automatic or mandatory step in accessing items sequentially, 
or otherwise continuing to interact with the system, without exiting the system, the 
interaction or some level of the interaction. 

20 Let 




We will also refer to the columns of Gj as & 
Let 

and 

~ rn r 12 • • ■ r lp 
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We will also refer to the rows of R as ri > r2 > ■ • ■ > t p and the columns of R as 
r 1 , r 2 , . . . ,r p _ 

4.2 Number of Ratings 

Let 

{0 if user i = user j 
1 if user j has assigned a rating to the n th item contributed by 
0 if user j has not assigned a rating to the item 



Ki = 

(PXCx) 



We will also refer to the columns of Ki as k h k h--- > k i'- 
Let 

so that 

0 < Uj < Ci 



hi '•■ : 
_ i p i • ■ • tpp 

We will also refer to the rows of Tas • ■ • > *p and the columns of ras 



60195430vl 



9^ 



5. Basic Expert Method 

The Basic Expert method determines h t the "Expertise" of a user, according 
to the ratings other users assign to items contributed by such user. 



In the initial embodiment of the method, Expertise is the arithmetic mean of 
the ratings that other users assign to items contributed by user i: 



\ h d otherwise 
In the initial embodiment, the Expertise of each user is calculated periodically, once 

10 every 12 hours. 

5.2 Default Expertise 

A user who has not contributed any items, or none of whose contributed items 
has been rated by any other user, is preferably granted the arithmetic average 
Expertise of users whose contributed items have received ratings: 

15 



5" 



5.1 



Initial Embodiment 





Note that, in the initial embodiment, the first time any item contributed by a user 
receives a rating, the Expertise of the user is set equal to that rating. 
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6. Basic High Regard Method 

The Basic High Regard method determines h i , the "Regard" of a user, 
according to the ratings that other users assign to items contributed by user i and the 
5 Regard of the users providing the ratings. 

6 . 1 Initial Embodiment 

Fig. 7(a) is an overview of data flows in an example High Regard ratings 
system, showing the circular nature of Regard affecting the Regard of other users and 
1 0 the Quality of items contributed by other users. 

As shown, one or more users contribute one or more items (also called 
content). A user does not have to contribute any item, but if he or she does contribute 
at least one item, the other users are given the opportunities to view and rate each item 
contributed. Each item is rated (for example, arrows 704, 709, 707) by one or more 
15 users. For example, as shown in Fig. 2(a), the user may rate a current message in an 
online forum. As another example, as shown in Fig. 2(c), the user may rate a web 
page. Various other types of items and groups of items are discussed herein. 

The ratings of the other users, along with the Regard of the users providing the 
ratings are used to create a Quality value of an item. Meanwhile, the Regard rating of 

20 the users giving the ratings are potentially being changed if those users are 
contributing items of their own or other users are rating their previously contributed 
items. As discussed below, not all embodiments determine Quality immediately when 
an item is entered. In a preferred embodiment, the Regard of users is updated 
periodically, for example, every twelve hours, although a shorter period of time can be 

25 used. In the future, as computing power increases, it is contemplated that both 
Quality and Regard will be updated more frequently. 

Fig. 7(b) is a more detailed example of Fig. 7(a). It will be understood that the 
"matrices" shown here are a conceptual presentation of a described embodiment of the 
present invention. An actual implementation of the matrices shown in Fig. 6(b) is 

30 discussed below in connection with the data structure of Fig. 1 1 . The rating of each 
item 712 contributed by each user and rated by each of the other users can be stored in 



601 95430v 1 



a database as the record of ratings 718. The ratings 714 given to an item by the users, 
together with the Regard values 716 of the users, can be used to derive a measure of 
the Quality of the item 712. The Regard values of the users, are created by using the 
entire record of ratings 718. In some embodiments, an iterative method 714 can be 
used to simultaneously create the Regard values of the users and to use the Regard 
values to weight the ratings given by the users in the record of ratings. 

Once the Regard values 716 of the users are determined, one can then 
calculate the Quality 720 of an item as discussed below in Section 5 et seq. 

Certain embodiments calculate user Regard and item Quality values per user 
demand. Other embodiments calculate these values periodically and store them in a 
database for later retrieval. 

In the initial embodiment of the Basic High Regard method, Regard is the 
arithmetic mean of the ratings that other users assign to items contributed by user i, 



weighting each rating by the Regard of the user providing the rating: 

The Regard of each user is potentially an input into the calculation of the 
Regard of every other user, in addition to item ratings. Therefore, the High Regard 
method involves the simultaneous solution of a number of related equations. 

In the initial embodiment, the Regard of each user is calculated periodically, once 
every 12 hours. 

6.2 Default Regard 

A user who has not contributed any items, or none of whose contributed items 
has been rated by any other user, is granted the arithmetic average Regard of users 
whose contributed items have received ratings: 
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Note that, in the initial embodiment, the first time any item contributed by a user 
receives a rating, the Regard of the user is set equal to that rating. 



6.3 Matrix Notation 



The interdependency of each user's Regard can be expressed in matrix 
notation. Setting: 



«4l 
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= Th h = 
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. tpi 
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ifa?#0 
\h b d otherwise 
ff Xa b ^0 
\h b d otherwise 

1 h b d otherwise 



7. Basic Quality Method 

The Basic Quality method determines 1? , the "Quality" of an item, according 
to the ratings assigned to the item and the Expertise or Regard of the users providing 
the ratings. 



7 . 1 Initial Embodiment 



In the initial embodiment of the method, Quality is the arithmetic mean of the 
ratings that other users assign to item n contributed by user i, weighting each rating by 
the Expertise or Regard (the method being selected by the system operator) of the user 



^ if£*£#o 
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providing the rating: 



In the initial embodiment, Quality is calculated on demand — at the time it is needed 
for any purpose — preferably subject to, a time period following the most recent 
calculation, for example, a 10 minutes during which time a cached copy of the most 
recently calculated Quality value for the item is provided. 

7.2 Default Quality 

An item which has not received any ratings is granted the arithmetic average 
Quality of items that have received ratings. 



EE 



EE 



0 otherwise 



i ifE^-^o 



0 otherwise 



Note that, in the initial embodiment, the first time an item receives a rating by any 
user, the Quality of the item is set equal to that rating. 



7.3 Matrix Notation 

The calculation of Quality can be expressed in matrix notation: 



n _ ^fl? 

(lxi) hkf 
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8. Analysis and Illustration 
8. 1 Basic Expert Method 

An assumption of the method is that an item rating is an effective 
measurement of the capacity of the user who contributed the item to identify valuable 
items contributed by other users. From the point of view of the user assigning item 
ratings, the user is helping to identify experts whose assessment of the value of items 
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will receive higher weighting. From the contributor's point of view, high item ratings 
mean higher Expertise, implying more influence over Quality levels. In this sense, the 
Basic Expert method is an expert system. 

Expertise under the Initial Embodiment 
5 In a universe of four users i,j, x and z, with user i having contributed at least 

one item and users j, x and z having each rated at least one user i item: 

7 _ Tjj + r„ + r iz 
~ t l3 +t tx + t iz 

In the calculation of Expertise, item ratings are weighted equally, Regardless of which 
users assigned the ratings. 
10 Partial differential equations demonstrate the impact on user z's Expertise of an 
additional item rating by user j: 

rating/Expertise differential 
dhj _ ff£ ~ hi 

dtij ~ Uj + t ix + tf, 

total # of previous item ratings 



|i>oif sS >^ 

User z"s Expertise increases whenever another user rates an item contributed by user / 
15 above user z"s previous level of Expertise. The increase is equal to the difference 
between the rating and the previous level of Expertise, divided by the total number of 
ratings previously assigned to user i items. 

Quality 

If users x and z have already assigned ratings to item n contributed by user /: 

^ = h x gj x + h z g? 
' h x + h z 

20 

The impact on item Quality of a rating by user j is: 

rating/Quality weighting by 
diffe rential ra ter^ Expertise 

dtij h x + h z 

total Expertise of 
previous raters 
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The Quality of item n contributed by user i increases whenever another user rates the 
item above its previous level of Quality. The increase is equal to the difference 
5 between the rating and the item's previous level of Quality, weighted by the Expertise 
of the user providing the rating, divided by the total Expertise of users who previously 

rated the item. 

8.2 Basic High Regard Method 

An assumption of the method is that an item rating is a useful measurement of 
10 the capacity of the user who contributed the item to identify talented item 
contributors. 

From the point of view of the user assigning item ratings, the user is helping to 
identify experts whose assessment of the value of other users' items will further refine 
the selection of experts. From the contributor's point of view, high item ratings mean 
15 higher Regard, implying more influence over item Quality and more influence over 
the Regard of other users. In this sense, the Basic High Regard method is a robust 
expert system. 

Regard under the Initial Embodiment 

Again in a universe of four users, with user i having contributed at least one 
20 item and users j, x and z having rated at least one item contributed by user i: 

b Tijh) + r lx h b x + r iz h b z 
* ~ kM + tixhi + t iz h\ 



In the calculation of Regard, the sum of all the ratings that each user has assigned to 
items contributed by user i is weighted by the rating user's Regard. 

The impact on user f's Regard of an additional item rating by user j is: 

weighting by 

rating/Regard rater » a Regard cross-rating dependencies 

differential t *~ ' 

dhi l^W) ft + g*) + fr« - + (r» - + (r. - ^)gj 

dtij Ujh) + t ix h b x + t iz h b 2 

total weighted # of previous item ratings 
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User z"s Regard increases whenever another user rates an item contributed by 
user i above user z's previous level of Regard. 

The increase is equal to the difference between the rating and user z's previous level of 
5 Regard, weighted and adjusted, divided by the total weighted number of ratings 
previously assigned to user i items. 

£>oiffl5>tf 

The weighting in the numerator is the rating user's Regard. The adjustment for cross- 
rating dependencies accounts for the effect of the additional rating on the Regard of 
users other than i — and hence, how the rating affects the weight assigned to their 
10 previously assigned ratings of user z items. Cross-rating dependencies can magnify or 
diminish 
ah, 

, but cannot change its sign. 
Quality 

If users x and z have already assigned ratings to item n contributed by user z: 

n n _ h b x g? x + h b z gf z 
1 h b x + h\ 

15 

The impact on item Quality of a rating by user j is: 

weighting: 

rating/Quality rater's Regard cross-rating dependencies 

differential ' , * ' . "* 

dtij h b x + h\ 

total Regard of previous raters 

^ > 0 if g» > h\ 

The impact of a rating on item Quality is similar in form to the impact of a rating on 
the contributor's Regard. The Quality of item n contributed by user i increases 
20 whenever another user rates item n above the item's previous level of Quality. 

The increase is equal to the difference between the rating and the item's previous level 
of Quality, weighted and adjusted, divided by the total Regard of the users who 
previously rated the item. 

The weighting in the numerator is the rating user's Regard. The adjustment for cross- 
25 rating dependencies accounts for the effect of the additional rating on the Regard of 
users other than i — and hence, how the rating affects the weight given their 
previously assigned ratings of item n. Cross-rating dependencies can magnify or 

diminish ^ , 
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but cannot change its sign. 



8.3 Rating Dependencies 
Basic Expert Method 

An item rating can affect the Expertise of the user who contributed the item, but 
not the Expertise of any other user. However, a rating of an item contributed by one 
user can affect the Quality of an item contributed by another user. A simple example 
of this is discussed above in connection with Fig. 6. 



• user j rates item n: 

- above item n's Quality — » Quality rises 

- above user i's Expertise ->• Expertise rises 



• user i's previous rating of item m: 

- above item m's Quality -> Quality rises 

- above user x's Expertise — ► Expertise unchanged 



Table 1 . Basic Expertise: Rating Dependencies 



In the example case, a rating of item n contributed by user i can affect the Quality of 
item m contributed by user x (already assigned ratings by all users): 



x (m) 



rating dependencies 



df, i0 




hi + hj + h z 



total Expertise of users 



where 



dhj 
dtij 



9ij ~ hi 



y£>0ifg§>h i andg5>q? 
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When user j rates item n above user /'s previous level of Expertise, user z"s Expertise 
rises. If user i has already rated an item m above its previous level of Quality, then 
item m's Quality rises. 

Thus, an item rating potentially affects the Quality of the item, the Expertise of 
the contributor and the Quality of every item ever rated by the contributor. A rating 
has secondary effects that extend as far as one other user and that user's rating 
activities. The method recognizes an item rating as, in part, an indication of the 
ability of the contributor of the item to identify valuable items contributed by other 
users. 

Basic High Regard Method 

An important distinction of the Basic High Regard method is that one user's 
rating of an item contributed by a second user can affect the Regard of a third user. 



I 

x (m) 

• user j rates item n: 

- above item n's Quality -> Quality rises 

- above user z's Regard Regard rises 



• user i's previous rating of item m: 

- above item m's Quality Quality rises 

- above user x's Regard — ► Regard rises 



Table 2: Basic High Regard: Rating Dependencies, Sparsely Rated Items 

Thus, the method recognizes an item rating as, in part, an indication of the ability of 
the contributor of the item to identify other talented items contributors. The rating of 
an item by a contributing user affects the Regard of other users in the system. 
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3 (p)&1 * (") 

IT & IT 

z (o) ±=5 x (m) 

• user j rates item n: 

- above item n's Quality ->■ Quality rises 

- above user i's Regard — > Regard rises 



• if all the users have rated each other's listed 
items, the Regard of each user and the Quality 
of each item may change. 



• as the Regard of users change, their previous rat- 
ings of user j and item n may receive greater or 
lesser weight, circling back to magnify or dimin- 
ish, hut will not entirely offset, the impact of user 
fa rating. 



Table 3: Basic High Regard: Rating Dependencies, Fully Populated 
An item rating has ripple effects that can propagate through the entire network, 

potentially affecting the Regard of the user who contributed the item, the Regard of 

the user who assigns the rating, the Regard of other users, the Quality of the item and 

the Quality of other items. 

A useful example is a situation in which cross-rating dependencies are limited 

to two parties. For example, if users j and z have previously assigned item ratings that 

are equal to the Regard values of the contributors, cross-rating dependencies are 

limited to users / and x. 

In this case, the formulas for the impact of an additional rating by user j on 

user z's Regard reduce to: 

cross-rating dependency 

rating/Regard weighting by 

„ , between users 1 and x 
differential rater's Regard , 



9hi Qfi-»t) *'(*}) + fr--'fr->g* 

dUj Ujh) + t ix h b x + t iz h\ 

total weighted # of user i item ratings 



cross-rating dependency 
between users i and x 

dtij t X ih\ + t X jh b j + t xz h b z 

total weighted # of user x item ratings 
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Thus, if the rating is above user i's previous level of Regard, then 



9ij -h\>0 

and, because cross-rating dependencies can diminish, but not reverse, the impact of a 
rating 

5 

If user i has rated items contributed by user x on average higher than user x's Regard, 
then 

r xi - h\t xi > 0 

and 

— > 0 
dtij 

10 

If user x has rated items contributed by user i on average higher than user i's previous 
level of Regard, then 

r ix - hUix > 0 



Hence, the impact of the rating is magnified; i.e., it exceeds the rating/Regard 
20 differential weighted by user fs Regard. 

The alternatives can be presented in tabular form: 
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Magnified or 
Diminished 



XT } Cross-Rating Dependency 



Average user i rating of user x items 



Average user x ratings of user i items 


above Regard 

(r xi - h b x t xi > 0) 


below Regard 

(r si - h b x t xi < 0) 


above Regard (r ix - h% x > 0) 


Magnified 


Diminished 


below Regard (r ix - Jijt« < 0) 


Diminished 


Magnified 



Table 4: Impact of Cross-Rating Dependencies on Regard (Outsider Rating) 



This result also applies to larger numbers of users with a populated data set. 
When users owe their Regard, in part, to cross-rating each other — or have mutually 
reduced each other's Regard — their Regard is subject to greater volatility under some 
circumstances. A rating by a user without a cross-rating dependency, such as user j in 
the example, will have greater impact than suggested by the user's Regard alone. 

A similar result holds when one user rates an item contributed by another user with 
whom the user shares a cross-ratings dependency. 



Magnified or i 

Diminished { [j]t } Cross-Rating Dependency 
x (m) 



Current (item m) + Average user i 
ratings of user x items 



Average user x ratings of user i items 


above Regard 

(9£+r*,-(l+/4U>0) 


below Regard 
(ff3»"„ - (1 + h b x t xl ) < 0) 


above Regard (r ix - > 0) 


Magnified 


Diminished 


below Regard (r ix - h b t t ix < 0} 


Diminished 


Magnified 



Table 5: Impact of Cross-Rating Dependencies on Regard (Cross-Rating Party Rating) 



This aspect of the High Regard method facilitates construction of user 
"communities". Users who associate and build a web of positive cross-ratings can 
help each other increase their Regard quickly — the effect of their positive ratings on 
each other's Regard is magnified. 
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This aspect of the described method also heightens the risk of mutual negative 
ratings, in the course of a form of destructive behavior often observed on wide area 
networks, known popularly as "flaming" — intentional provocation or repetitive 
attacks, typically in the form of an exchange of written messages. Two or more users 
5 engaging in mutual attacks lose Regard quickly — the effect of their negative ratings 
on each other's Regard is magnified. 

Ratings assigned by "outsiders", users outside the web of cross-ratings, can 
have similar effects. An outsider who boosts the items of members of a community 
10 can prompt a magnified increase in their Regard. An outsider who rates negatively 
the items of users engaging in flaming can cause a magnified drop-off in their Regard. 
However, these magnifying effects do not circle back to affect the outsider's Regard, 
because there are no cross-rating dependencies as to the outsider. 

15 Users who associate in communities or engage in flaming, who subsequently 

reverse course and rate each other in the opposite direction, have a diminished impact 
on Regard. In fact, any time that two users or groups of users rate each other's items 
in opposing directions, it creates conflicting cross-rating dependencies. In effect, a 
user is either weakening one of the sources of the user's own Regard, or strengthening 

20 a force that is acting against the user's own Regard. 



25 



Magnified or 
Diminished 

j -» i -> x 
z 



Table 5: Chain of Dependencies 

30 Similar overall effects can arise when, instead of direct cross-ratings, there is a chain 
of dependencies. Whether the original rating is magnified or diminished is a question 
of the product of the signs of these links in the chain. For example, if user i has rated 
items contributed by user x above user x's Regard (+), user x has rated items 
contributed 

35 by user z below user z's Regard (-) and user z has rated items contributed by user i 
below user z"s Regard (-), then the product of the signs is 

(+) x (-) x (-) = (+) 

In this case, user fs rating of user i is magnified. 
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Cross-Rating Dependencies — Quality 

Deriving Quality in the same two-user cross-ratings scenario triggers the same 
cross-ratings effects. 

The formulas for the impact of a rating by user j of item n contributed by user i reduce 



cross-rating dependency 

differential rater's Regard 



rating/Quality weighting by ^ . and x 



total Regard of previous raters 



cross-rating dependency 
between users i and x 

dUj t xi h\ + t xj h) + t xz h b 2 

total weighted # of user x item ratings 



Thus, if the rating is above item n's previous level of Quality, then 

and, because cross-rating dependencies can diminish, but cannot reverse, the impact 
of a rating 




If user i has rated items contributed by user x on average higher than user x's Regard, 
then 



and 
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r X i - Kltxi > U 



If user a: has rated item n contributed by user i above its previous level of Quality, then 

- Q? > 0 

and 

5 

Hence, the impact of the rating is magnified; i.e., it exceeds the rating/Quality 
differential weighted by user fs Regard. 

1 0 The alternatives can be presented in tabular form: 

|£ Magnified or 
Diminished 

j E>] i(n) 

it }Cross-Rating Dependency 

x 



Average user i rating of user x items 



User x rating of user i's item n 


above Regard 

ir xi - h\t xi > 0) 


below Regard 

(r xi - h\t xi < 0) 


above Quality ($£. - qf > 0) 


Magnified 


Diminished 


below Quality (g? x - qf < 0) 


Diminished 


Magnified 



Table 7: Impact of Cross-Rating Dependencies on Quality 

15 

9. Extended Methods 
9.1 Expert Methods 

A general formulation for alternative embodiments of the Expert method 
20 (the "Extended Expert methods") is: 

h e d otherwise 
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9.2 High Regard Methods 



A general formulation for alternative embodiments of the High Regard 
Method (the "Extended High Regard methods") is: 



if EEA^O 

j=ln=l 



A general formulation for alternative embodiments of the Quality method (the 
"Extended Quality methods") is: 




g d otherwise 



1 0 9.4 Transformation Function 

®(' x ) is a transformation function and *( x ) is a function that reverses the effect of 

\I>($(a;)) = re 

15 Note that if 

&(x) = a; 



<b(x) = x 



20 



then the Extended methods calculate arithmetic means — weighted, in the case of 
Regard and Quality. In this case, the Extended Expert method is equivalent to the 
Basic Expert method, the Extended High Regard method is equivalent to the Basic 
25 High Regard method and the Extended Quality method is equivalent to the Quality 
method. 

If 

= log{x) 
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and 



then the Extended methods calculate geometric means. 

\I>(x) = exp(x) 



5 If 



= - 

X 

and 

x 

then the Extended methods calculate harmonic means. 
If 

<3>(a;) = x n 



and 



where n is a parameter set by the system operator, then the Extended methods 
calculate Holder means. In alternative embodiments, n may be set to any real number. 
In one embodiment, n=3. In another embodiment, n=l/3rd. 



9.5 



Default Value 



In alternative embodiments, the Expertise or Regard granted a user who has not 
contributed any items may be determined in accordance with: 



in which K represents the Expertise or Regard, as the case may be, of user /; or, more 
generally, an amount otherwise related to the ratings assigned to items contributed by 
other users: 
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h e d = tf(T, R) 



or simply set to zero: 



0 



The Quality granted an item to which no user has assigned a rating may be determined 
5 in accordance with: 



or, more generally, an amount otherwise related to the ratings assigned to other items: 



9.6 Periodicity 

In alternative embodiments, the Expertise or Regard of each user, as the case may be, 
is calculated periodically, after the passage of a specific period of time, after the 

15 collection of a specific number of new item ratings, or as often as possible given the 
available computational resources. In alternative embodiments, the Quality of each 
item is calculated every time the item's Quality is requested or required, subject to a 
minimum time period following the most recent calculation, during which time a 
cached copy of the Quality value most recently calculated is provided; or after the 

20 passage of a specific period of time, after the collection of a specific number of new 
item ratings, or as often as possible given the available computational resources. 




q d = e{q\...q[ 1 ,... 



or simply set to zero: 



g d = 0 



10 



9.7 Matrix Notation 



The interdependency of each user's Regard in the Extended High Regard 
25 method can be expressed in matrix notation. 



Let 
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4? 



Setting: 



R*-- 









= R*h e = 







h% 



°1 






tn 




= Th e 






- a l ■ 









tip 



- h\ ■ 

h% 



ifa! # 0 
^ otherwise 

if al ^ 0 
ft5 otherwise 

f*(^|) ifa^O 
[ ft^ otherwise 



The calculation of Quality can also be expressed in matrix notation: 

15 



h\ h\ 



10. Computational Method 



10.1 Description 

The following procedure solves for Expertise h t in one step and for Regard h 1 by an 
iterative method of successive approximation. 

10.2 First Step 
Let 



h d otherwise 
h d otherwise 



if £ty*0 
ise 

if jzt 2j ± 0 



if e*pj^o 



/id otherwise 



10 where is a selected transformation function: 
rS = E*(*5) 



and ^ is determined according to the selected procedure for calculating a default 
value. In one embodiment, in this first step: 



In each case, using a function ^( x ) , for which: 

Stopping here results in h t , Expertise under the Extended Expert method; and if: 

&(x) = X 

then M is equivalent to h , Expertise under the Basic Expert method. 
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Second Step 



Calculation of Regard requires additional steps. 

A first approximation of each h t value is calculated as follows: 



= R*h e = 



'11 '12 

r* r* 
'21 '22 



^11 tl2 
*21 ^22 



[ h^ 1 otherwise 
| hj 1 otherwise 



otherwise 



where h i is the Expertise of each user calculated above and n d is determined 
10 according to a selected procedure for calculating a default value. In one embodiment, 
in this second 

step, the value is based on the K values determined in the first step. 



0 otherwise 
1 if fjij # 0 



Using a function , for which: 

*(*(*)) = 
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10.4 Subsequent Steps 



The following procedure is repeated successively with m=(2,3,4, - ), converging 
toward a solution. 







r ll r 12 








U)2 m 


= R*h e "'- 1 


r* r* 
'21 '22 










■ W P 




L r P i 


r* 




Mr 






hi *12 - - ■ 


tip" 




' h 














h 




■ V - 




.hi ■■• 


£pp . 




. " h 





J vf(ffer) if af™ # 0 
1 /i^" 1 otherwise 
j*(f£) if # 0 
1 h e r otherwise 



tt(lfcr) if a? * 0 
otherwise 



10 where h cT is determined according to a selected procedure for calculating a default 
value. In one embodiment, in this second step, the value is based on the h i values 
determined 
in the previous step. 

0 otherwise 



0 otherwise 



Using a function ^( x ) , for which: 



($(*)) = a; 



lim (A-™) = /if 
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The procedure converges toward M , Regard under the Extended High Regard method 
and if 



$(ar) = x 

5 then h t is equivalent to h i , Regard under the Basic High Regard method. 
Convergence 

This procedure can be applied successively a specified number of times m. In one 
embodiment, m=5. In other embodiments, the procedure can be applied successively 
until the approximation of Regard at the beginning and at the end are nearly equal. In 
1 0 one embodiment, the procedure is considered complete when: 

| max I? (A?™ -A- r "" 1 )| < -005 

1 1 . Inertia 

11.1 Expertise/Regard Inertia 

1 5 Under the Expert and High Regard methods, the first time that any of the items 

contributed by a user receives a rating, the user's Expertise or Regard, as the case may 
be, is set equal to that rating. This could have undesirable consequences. The items 
contributed by a user may be rated initially by a poorly Regarded user and for a period 
of time thereafter, by a small or poorly Regarded subset of users. 

20 In order to control the amount by which item ratings influence the Expertise or 

Regard of relatively new or infrequent users, we create a notional user i=l, with 

rn = a{t, ti, r l , Ti, h e ) 

and 

Ui = 0(f, U, r\ t u h e ) 

25 where h e denotes the vector of Extended Expertise or Regard values, as the case may 
be, for all users. 

Thus, the notional user assigns ni and U\ for each user, taking into account 
the number and level of ratings assigned to user z's items by other users, the number 
and level of ratings user i has assigned to items contributed by other users and the 
30 Expertise or Regard of all users. 

In certain embodiments, r n is set equal to and U\ is determined 

according to an equation in the form of: 

a — (bx + cy — y/dxy) e 
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where a,b,c,d and e are constants selected by the system operator and x represents the 
aggregate number of the user's items that have been rated by other users and y the 
number of other users' items that have been rated by the user. Using a relatively high 
5 value for a and small (positive) values for b,c,d and e, this formulation has the 
characteristic of reducing the fixed constant a more slowly when x and y grow in 
tandem and more quickly when x and y grow divergently. 

This property causes the system to withdraw inertia as a user interacts with the 
system, but at different rates: more slowly from a user who rates items and whose 
10 items are rated by other users close to a targeted ratio and more quickly from a user 
who primarily does one but not the other. 



Fig. 8 is a three-dimensional plot of a High Regard Inertial Model having two 
axes labeled from 0 to 100 representing, along the first axis, the number of ratings 
other users have assigned to items contributed by the user in question and, along the 
second axis, the number of ratings the user in question has assigned to items 

20 contributed by other users. A third time axis labeled from 0 to 500 represents the 
number of notional ratings (set at a predetermined default level of Regard) assigned to 
an item contributed by the user in question in order to create the inertia effect. As can 
be seen, the effect of inertia diminishes as other users assign ratings to items 
contributed by the user in question and as the user in question assigns ratings to items 

25 contributed by other users. The shape of the three-dimensional plot represents the 
characteristic of the specified embodiment that the inertia effect diminishes more 
slowly when the user in question rates items contributed by other users and the user in 
question's contributed items are rated by other users, in tandem. The described 
embodiment reflects a target ratio. 

30 The Regard inertia effect in the described embodiment diminishes rapidly, in 

contrast, when the user in question rates items contributed by other user often, but few 



Example 



r 5 i = tbih e d 



t 5 i = max{0, 500 



y n=2 n=2 \ n=2 




15 
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users rate items contributed by the user in question, or vice versa. In the described 
embodiment, the number of notional ratings is initially 500. 

11.2 Item Quality (Quality inertia) 

5 Under the Expert and High Regard methods, the first time that any item 

receives a rating, the item's Quality is set equal to that rating. This could have 
undesirable consequences. The item may be rated initially by a poorly Regarded user 
and for a period of time thereafter, by a small or poorly Regarded subset of users. 

In order to control the amount by which early item ratings influence the Quality of 
10 items, we create a notional user i=2, with 

9a = 7(9. d, fc", 9i, h) 

and 

where d represents the number of units of time, using a standard unit selected for a 
preferred embodiment (e.g. minutes, hours or days), measuring the time elapsed since 
15 user z contributed item n. In the following example, d is expressed in minutes. 

In other words, the notional user assigns a 9a and k a for each item, taking into 
account the Quality of other items with a track record of ratings, time elapsed since 
the item was contributed and the number and level of ratings assigned to the item by 
other users. 

20 In certain embodiments, 9a is set equal to , with the formula determining k a in 
the form of: 

a — (bx + cy — \/dxy) e 

In this case, x represents the aggregate number of ratings assigned to the item and y 
the time interval d. This formulation causes the system to withdraw inertia as users 
25 rate the item and with the passage of time, but at different rates: more slowly for an 
item that receives ratings at a targeted rate per period of time and more quickly for an 
item that receives ratings at a higher rate (resulting in adequate data) or at a lower rate 
(suggesting disinterest). 

Example 



9fj = k? 2 q e d 



Aft = max{l, 600 - 




30 
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Fig. 9(a) is a three-dimensional plot of a Quality Inertia Model having a first 
axis labeled from 0 to 300 units of time as specified above, a second axis labeled from 
0 to 80 representing the number of ratings users have assigned to the item in question 
and a third time axis labeled from 0 to 600 representing the number of notional ratings 

5 (initially set to a predefined default level of Quality) assigned to the item in question 
in order to create the inertia effect. As can be seen, the effect of a rating diminishes 
with the passage of time and as other users assign ratings to the item in question. The 
shape of the three-dimensional plot represents the characteristic of the described 
embodiment that the inertia effect diminishes more slowly when users rate the item in 

10 question steadily in small numbers. The described embodiment reflects a target 
number of ratings per units of time. The Regard inertia effect diminishes rapidly, by 
contrast, when considerable time passes with very few ratings contributes, or if a large 
number of users rate the item during a short time interval. 

15 11.3 Clarification 

Notional user values created for calculations of Quality will, in some 
embodiments, be excluded from the calculation of High Regard values. 

12. Decay 

12.1 Description 

20 In order to keep Regard and Expertise values current, or to discount older or 

unusual historical results, some embodiments will include "Decay", procedures for 
adjusting the record of ratings with the passage of time. 

12.2 Simple Time Decay 

In one embodiment, a standard time period, e.g. days, weeks or months, 
25 determines the rate at which ratings are removed from the calculation. 

Let g be the number of such standard time periods from which ratings will be included 
in the calculation of Expertise or Regard, as the case may be, after which they are 
removed. 

Let f m and R m be the (p x p) matrices of *y and r v values reflecting only the 
30 ratings provided during a single period m (Regardless of the date the item was 
contributed). T m and Rm , defined below, substitute for T and R to calculate Expertise 
or Regard, whichever is used in the embodiment. 

For m=l, let 
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and 



5 



For 2 < rn < g 
and 

For m > 9 
and 



Rm = R m . 

Rm = Rm-1 + R" 1 - 



Rm = Rm-1 - R m ~ S + R" 1 

12.3 Segmented Decay 

Other embodiments of Decay treat differently subgroups of users segmented 
according to one or more user characteristics. In some embodiments, it is helpful to 
divide the users into a number of subsets and to use the method of "segmented decay" 
to alter the number and value of ratings among the record of ratings for items 
contributed by particular groups of users, applying systematic but different procedures 
to each subset. Users can be divided into subsets according to their Regard (or 
Expertise) values, or some other appropriate attribute. For example, one can make the 
ratings given to items contributed by the most poorly Regarded users "sticky," that is 
largely unaffected by decay procedures, or even strengthen the worst ratings under 
some circumstances. In this way, users with the lowest level of Regard who are prone 
to disruptive behavior are largely and permanently excluded from user interactions. 
Alternatively, one can make the worst ratings given to items contributed by both well 
Regarded and poorly Regarded users decay faster over time than for users who have 
more average Regard values. In this way, well Regarded users have an incentive to 
make continuous contributions to stay highly ranked and poorly Regarded users will 
receive encouragement and assistance to improve their Regard. Additionally, one can 
use Decay to encourage users to contribute ratings of items contributed by other users, 
for example, by causing the better ratings of items contributed by well Regarded users 
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to persist longer, if such users actively rate items contributed by other users. The 
following paragraphs describe an embodiment of segmented decay, as expressed 
mathematically. 



Let d ( h > u ) be a function that maps users into subsets u i > • • • > U z . In some 
embodiments, this function will segment users according to their previously computed 
Expertise or Regard. 

A standard time period, e.g. days, weeks or months, determines the rate at which 
ratings are affected by a transformation function using the procedure set forth below. 

Let T m and R m be (p*p) and d*p) matrices including only ratings provided during 
period m (by any user in any subset) for items contributed (at any time) by users in 
subset 0x . 

To compute either Expertise or Regard with segmented Decay, substitute: 
and 

R = Rm 



using T m and defined below. 

e m e * 
T m = ^5 T {l,m,x,e x ,T l ) 
1=1 

A transformation function 5 is selected, such that 
and 



Example 

R\^Y^5 R {l,m,x,e x ,R\T l ) 
1=1 

User % e Ul if 0 < hi < 0.25 
In one embodiment, e i h ^ u ) separates users according to four levels of Expertise or 
Regard 



User i € C/ 2 e if 0.25 < h { < 0.50 
User i e Ul if 0.50 < hi < 0.75 
User i e U e A if 0.75 < h { < 1.00 



60195430vl 



The procedure adjusts the record of ratings as follows. When a user provides a rating 
of an item contributed by another user, the rating remains in the record unaffected 
during the following 10 standard time periods. All such ratings are removed from the 
record after 25 

5 standard time periods. Between the 10 th and the 25 th period, T and R are 

adjusted (according to different formulas for each subgroup) which lessen the impact 
of each rating as the periods progress and adjust older ratings from lower values to 
higher values in the case of poorly Regarded users and from higher values to lower 
values in the case of well Regarded users. 

10 The procedure continues to base Expertise or Regard, as the case may be, on 

item ratings assigned over an extended period of time, but lightens the weight of bad 
historical ratings on poorly Regarded users and reduces the benefit of good historical 
ratings for highly Regarded users. These effects help users with poor ratings to 
recover Expertise or Regard and encourage users with high ratings to make additional 

1 5 contributions. 

For all subsets ®* e #2, 0 3 , 9 4 } ? wn en m=l, let 



For 2 < m < 10 



T m = T 1 + ■ • • + T m 



j2 TO = .R 1 + -.- + iz m . 



For 11 < m < 25 




For rn>26 



Rm — R m + - — I- R m - 10 + pfiT" 1 " 11 + • 
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The following formula reduces proportionately the impact of ratings over the Decay 
period: 

for all Ox 



The following formulas cause ratings of each subgroup to move closer to the mean 
over the Decay period 



pf = pT 



. m _,x(J5)C'-") 



Pf = 1-Pf 



Fig. 9(b)) is a graph of a Segmented Decay. In accordance with the mathematical 
10 model, Fig. 9(b) displays the different adjustments made to ratings assigned to items 
contributed by users in four different categories of Regard. On a scale from zero to 
one, the first group 0, represents users with Regard of greater than 0.75 to 1.0 
(inclusive). The second group 0 2 represents users with Regard of greater than 0.50 to 
0.75. The third group 9 3 represents users with Regard of greater than 0.25 to 0.50. 
15 The fourth group represents users with Regard from 0.0 to 0.25. The horizontal axis 
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represents time periods 11-25 following the contribution of any rating. The time 
periods can represent days, weeks, or months, or some other time intervals. Time 
periods 0 through 10 are not represented on the graph because in the described 
embodiment, segmented decay does not begin to affect any rating during the first ten 
5 periods following the contribution of the rating. The described embodiment causes 
ratings to regress toward the rating of 0.5, represented by the movement of the 
diamonds, triangles, circles and spades toward the mean given the passage of time. 

Expressed mathematically, this is a transformation of elements of the R 
matrix. The solid straight line represents the incremental reduction in the weight 
10 assigned to older ratings. Expressed mathematically, this is a transformation of 
element of the T matrix. 

13. Vouch/Discredit 
13.1 Description 

15 In the case of items that have recently entered the system and therefore 

received few ratings, a robust calculation of Quality will not be available. In this early 
period, the Expertise or Regard (whichever is used in the embodiment) associated 
with the user who contributed the item may be the primary indicator employed by 
other users to predict whether the item has value and is worth review. 

20 "Vouching" permits users to associate their Expertise or Regard, as the case 

may be, with an item contributed by another user, as if they had contributed the item 
themselves. The procedure is designed to help solve the sparse ratings problem by 
giving users with relatively high Expertise or Regard an incentive to identify good 
items contributed by users with relatively low Expertise or Regard and to use their 

25 Expertise or Regard to bring the item quickly to the attention of the community of 
users. The Vouching user's higher Expertise or Regard value is thereafter associated 
with the item, instead of the Expertise or Regard of the user who contributed the item, 
in procedures that filter, highlight, sort or otherwise evaluate items based on the 
Expertise or Regard of the contributor. As the item subsequently receives ratings, the 

30 user who Vouched for the item receives credit — or potentially loses Expertise or 
Regard — to some extent, as does the user who contributed the item. 

"Discrediting" does the opposite: it permits users to dedicate their Expertise or 
Regard, as the case may be, to impugn an item submitted by another user. It is 
designed to give users an incentive to identify bad items contributed by users with 
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relatively high Expertise or Regard, quickly alerting the community of users. A value 
equal to one minus the Expertise or Regard of the user who Discredits the item is 
thereafter associated with the item, instead of the Expertise or Regard of the user who 
contributed the item. As the item subsequently receives ratings, the user who 
5 Discredited the item receives credit— or potentially loses Expertise or Regard-but in 
an inverse relationship to the ratings received by the item. 

13.2 Initial Embodiment 

In the initial embodiment, when user j Vouches for the item contributed by 
10 user i, new columns are added to the and matrices, such that: 

kj +1 = k? 

and 

,£ +1 = (l_rf) 

15 where "=" indicates the establishment of a link with column n of the K t and G< 
matrices. The newly created (and the original) columns reflect future ratings assigned 
to the item. In some embodiments, Vouching may also link the newly created column 
to historical ratings. 

When user j Discredits the n^ 1 item contributed by user z, new columns are 

20 added to the and matrices, such that: 

k? +1 = Jfe? 

where indicates the establishment of a link with column n of the K % and G i 
matrices. The newly created columns reflect the mirror image of future ratings 
assigned to the item. In some embodiments, Discrediting may also link the newly 
25 created column to the mirror image of historical ratings. 

In this embodiment, a user is only permitted to Vouch or Discredit if the user's 
Expertise or Regard (whichever is used in the embodiment), is higher than the 
Expertise or Regard currently associated with the item; initially, this is the Expertise 
or Regard of the user who contributed the item. Each user is permitted to Vouch for or 
3 0 Discredit a particular item only once. 

The newly created columns would not be factored into the calculation of Quality. 
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13.3 General Expression 



In some embodiments, a user who Vouches or Discredits might receive only a 
partial link to ratings for the item, expressed generally in the case of Vouching as: 

5 and 

And expressed generally in the case of Discrediting as: 
# +1 =r(l-tf) 

and 

A? +1 =r(*r) 

10 

where 

0 < r < 1 

For example, r = 0.5 would simply divide values in two, so that the user Vouching for 
or Discrediting an item contributed by another user is less affected by ratings. 

15 13.4 Sequential Vouching 

In some embodiments, a user who Vouches or Discredits will be affected by 
ratings on the basis of how much the act raises or lowers the profile of the item, as the 
case may be. 

20 Let U in - {a,b,c,...} be the set of users who have, sequentially beginning with user 
a, Vouched for or Discredited item n contributed by user z, where 

n(hi) < h a , n(h a ) < h b , n{hb) <h c ... 

Q(hj) = hj 

where 

if user j Vouches for the item and 

Q(hj) = l-hj 

25 if user j Discredits the item. 

Let U m be the m th user in U iu . 
In the case of Vouching, let 

T i = h Ui ~ ^ 

and, where m>\, 

r m = hu m - &{hu m -x) 
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In the case of Discrediting, let 

7? = hi-(l- h Vl ) 

and, where m>\, 

r^ = Q(h Um _ 1 )-(l-h u J 
5 To include the effect of Vouching in the calculation of Expertise or Regard, let 



and 

To include the effect of Discrediting in the calculation of Expertise or Regard, let 

k«£ +1 = rA(Jb?) 

10 and 

sKr +1 s<(i- fl r)- 

13.5 Initial Discredit 

In some embodiments, under certain circumstances, users may be permitted or 
1 5 encouraged to contribute items that are anticipated to yield a negative response. For 
example, an item might be a link to a web site containing objectionable content, 
which the user wishes to bring to the attention of the community of users. In some 
embodiments, a specific forum designed for this purpose may be established, clearly 
separated from other forums. 

20 If such items were to be directly associated with the contributing user, there would 

be substantial disincentives for participation. Therefore, in some embodiments, the 
user contributing the item will have the option to Discredit the item upfront, such that 

Adjusted 

9% = (1 - 9?j) 

The adjusted rating would be included in calculations of Regard and expertise for the 
25 contributing user. In some embodiments, the adjusted value will also be used in the 
calculation of the Quality of the item. 

13.6 Additional Discussion of Vouching and Discrediting 
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Figs. 20(a) and 20(b) are, respectively, a graph and a table to aid in explaining 
the concepts of "Vouching" and "Discrediting" and how these concepts provide a 
solution to the "sparse ratings problem" that arises during the early life of an item. In 
the graph of Fig. 20(a), the vertical axis represents the Regard of an individual posting 
5 an item (e.g., a Regard of 0.400). This is the Regard that is associated with the item, 
for example, for use in the calculation of thread caliber (discussed below in Section 
15). 

In certain embodiments of the present invention, there is a period shortly after 
an article is posted during which any ratings received from users may not be deemed 

10 sufficient to determine a robust measure of item Quality. The term "inertia" in Fig 
20(b) refers to Quality inertia, which may obscure the value of the theretofore 
received ratings, because too few ratings have been received during the period shortly 
following the contribution of the item. During this period, not enough ratings have 
been received to be certain that Basic Quality method or other methods of calculating 

1 5 Quality, or a simple average, or any other measure, is representative of user or expert 
opinion. 

One can, during this period, simply rely on the reputation/Regard of the user 
who contributed the item as a proxy measurement of the value of the item. A 
preferred embodiment, however, permits a user with a higher level of Regard to step 
20 forward and "Vouch" for the item. If, for example, the Regard of the Vouching user 
is 0.600, the Regard associated with the item being Vouched also becomes 0.600. 

When a user Vouches for an item, the user is making the strongest possible 
statement in support of the item in question, backing up that statement with the user's 
reputation. Vouching for an item implies that the user would be willing to take the 
25 ratings offered by other people for the item as if the user had actually authored it. A 
user cannot, for example Vouch a poorly Regarded item half-way up to the user's 
Regard level. Vouching, as used in the described embodiment, is all or nothing, based 
on the Vouching user's Regard value. 

Discrediting is a similarly strong statement. When a user Discredits an item 
30 the user is making the strongest possible statement in opposition to the item in 
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question, aligning the full value of the user's reputation against the item. In fact, the 
user would be willing to take the opposite of whatever ratings are given by other 
users. This means that a highly Regarded user with, for example, a Regard of 0.900 
will, by Discrediting an item, reduce the Regard associated with that item to 0.100 
(i.e., reduce the associated Regard by 1.000-0.900). In the described embodiment, 
poorly Regarded users are able to Discredit items contributed by highly Regarded 
users. But because items can be Discredited only to the extent of (1 -Regard), a poorly 
Regarded user who Discredits and item will not reduce the Regard associated with the 
item by much. Note that, although Quality values are associated with items and 
Regard values are associated with users, the Regard of the author of a particular item 
is often associated with the item. As noted above, for example, in the case of articles 
that have received few ratings, the author's Regard is a helpful benchmark for the 
item's value. The concepts of Vouching and Discrediting allow a "special" Regard to 
be used in calculations involving certain items instead of the Regard of the user who 
contributed that item. As shown in the example in the figure, a user having a Regard 
of 0.250 associates this Regard with items by Vouching for the items. Similarly, the 
user gives a Regard of 0.750 to any item which he Discredits. This particular user 
cannot affect the Regard of an item that already has a Regard higher than 0.250 and 
similarly cannot affect the Regard of an item that already has associated with it a 
Regard of less than 0.750. 

As items are Vouched for and/or Discredited, the Regard of the contributing 
user, as affected by these procedures, becomes a robust prediction of the Quality that 
will ultimately be calculated for the item. Fig. 20(b) shows calculation of Quality 
values starting at time period 9. (Prior to time period 9, an inertia value preferably is 
used for Quality). 

The horizontal axis of Fig. 20(a) shows 15 time periods (1 through 15). 
During each time period, in this example, a user Vouches or Discredits the same item. 
As shown in Figs. 20(a) and 20(b), initially, a user having a Regard of 0.400 posts the 
item. During time periods 2-4, three users Vouch for the item, improving the Regard 
associated with the item to 0.850. During time period 5, a user having a Regard of 
0.400 Discredits the item, giving it an associated Regard of 0.600. Users continue to 
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Vouch and Discredit the item until time period 15, when a user Vouches for the item 
and improves its associated Regard to 0.900. 

In Fig. 20(a), the shaded areas represent the maximum of Regard and Quality 
(i.e., MAX(Regard, Quality)) for the item. As discussed above, a ratings system does 
5 not necessarily have robust information until a certain amount of time has passed 
(e.g., 9 time periods). The results of Vouching and Discrediting, however, provide a 
robust prediction of item Quality much earlier in the process, even when ratings are 
"sparse." 

In addition, sequential Vouching/Discrediting preferably allows the Vouching 
10 or Discrediting user to share in the ratings received for the item, but only to the extent 
that the user moves the current Regard value associate with the item. For example, 
the user Vouching in time period 7 will have the least share in the ratings since the 
user has moved the Regard the least (from 0.800 to 0.875). The user who Discredits 
in time period 12 will have the most significant share (in the opposite of the ratings) 
1 5 since he moves the Regard the most (from 0.950 to 0.750). 

14. Multi-Item Categories and Attributes 

14.1 Description 

20 In some embodiments, items will fall into separate categories, weighted 

differently in the calculation of Expertise or Regard. Factors considered in 
establishing separate categories will include the difficulty of creating the items, the 
effort required by other users to review the items and the urgency of the items. 
The relative weight attached to an item in calculations of Expertise or Regard will 

25 depend on its category and a number of item attributes that are particular to the 
category. 

14.2 Formulation 

Let ^i 11 designate the category of item n contributed by user i. Let b i 2 > ■ ■ ■ > & " m 
identify additional attributes of item n, specific to the item's category. 

30 Let the function 

r? = r(&?\... ,&r) 
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map the category and attributes of the item to an additional weighting factor for use in 
calculations of Regard and expertise. 



14.3 Applied to the Expert Method 



I h% otherwise 



14.4 Applied to the High Regard Method 



E Ei7 ft J*(s,") 



h e i = < [ E^Er^ffc,", J j=i«=i 
\ h e d otherwise 



14.5 Example 



Let 1 = 2 , indicating that item 5 contributed by user i is participation in a chat 
session. 





Attributes 


Category 




bf 




1 Discussion Forum 


Subject Matter 


Length 


New Thread? 


2 Chat Session 


Participants 


Total Time 


Moderator? 


3 Online Auction 


Bids Submitted 


Image Offered? 


Successfully Sold? 


4 Web Link 


No attributes; all receive the same, fixed 


r? adjustment 


5 Market Forecast 


Equity, Bond, Other? 


How Far Ahead? 


Accuracy 


6 Auction Forecast 


Category 


How Far Ahead? 


Accuracy 



Table 8: Attribute Function 



H is a function of: 

• the number of participants, & P = 15 

• the total session time in minutes, &f 3 — 45 and 

• whether or not user i served as a moderator, ^i 4 = 0 (indicating not a moderator, 
whereas =1 would indicate the user was a moderator). 



£52 £53 

I? = maz{^,.25} + nu*{^,.3}H 
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= 0.15 + 0.3 = 0.45 

15. Caliber Method 



15.1 Description 

Some embodiments will involve the transmission, display or evaluation of 
groupings of items of different Quality, contributed by users with varying Expertise or 
Regard. A good example is a threaded discussion list — two or more discussion 
group postings with a common subject and explicit relationships between messages, 
i.e. an indication of which messages respond directly to which other messages. 

One objective of the method is to present groupings containing better items 
first, without breaking apart the thread structure. Multiple threads are sorted among 
themselves, based upon a measurement taking into account characteristics of some or 
all of the contents of each thread. 

The Caliber method determines C z , the "Caliber" of an item grouping. 

15.2 Initial Embodiment 

In the initial embodiment of the method, Caliber is the grouping average of, 
for each item, the higher of Regard (of the user associated with the item) and Quality: 



where z identifies the grouping (e.g. a thread identification number), m the number of 
items contained within, K the Expertise or Regard of the user who contributed item n 
and Qn the Quality of the item. 



In sorting threads, some embodiments may permit a user to user select h min and 9mm , 
such that items below a certain level of Quality, or associated with a user of Expertise 
or Regard below a certain level, are excluded from the calculation of Caliber: 



C z = — max{/i^, q^} 



15.3 



Caliber Threshold 




where 




15.4 Extended Caliber 



A general formulation for alternative embodiments of the Caliber method is: 



L n=l 



5 



based on the same transformation functions discussed above. 



Applied to the Caliber threshold method: 



h n if h n > h m i, 
0 otherwise 




g n ifq n > q mm 

0 otherwise 



:»)] 



where 



™ J 1 ifh n > h min or q n > h r 
i=i 1 0 otherwise 



10 



16. Auction Method 

Some embodiments will involve the transmission, display and integration of 
Expertise, Regard or Quality into the operation of online auctions. 

The Auction method integrates Expertise or Regard, specifically, into an 
15 auction pricing mechanism. The objective of the method is to increase the integrity, 
fairness and efficiency of online auctions by establishing standards and a procedure 
for a seller to limit the field of users permitted to bid in an auction. 

Let s " be an Expertise or Regard threshold defined by user i, who is putting 
item n up for sale in an online auction. In the initial embodiment, user / selects s ? at 
20 or before the commencement of the auction. If no value is selected, the default is 
zero. 

s ? will establish the minimum Expertise or Regard required for a user to 
submit a binding bid qualified to participate in whatever auction pricing formula 
determines the winner of the auction and the closing price. 
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Users who do not meet the s i standard, in the initial embodiment, will still be 
permitted to place "record bids" which are bid levels that are recorded and reported, 
either to the seller alone or to the seller and other users, together with data on the 
Expertise or Regard of the user making the record bid, subject to whatever limitations 
5 are in effect on the release of price information pursuant to the auction model. For 
example, in a "second price" style auction, only the second highest bid is disclosed to 
other potential buyers and this standard might be applied to record bids. 

In the initial embodiment, a user placing a record bid will have the option of 
withdrawing the bid at any time before the event described below. 

10 The selling user will have the option at any time until the conclusion of the 

auction, or until a defined period prior to the conclusion of the auction, to reduce the 
value of s ? . Record bids submitted by users who fall within the Expertise or Regard 
threshold after the reduction will immediately be made effective, pari passu with 
already pending bids and the users will no longer be entitled to withdraw. 

1 5 Importantly, although the selling user may reduce s ? , the selling user is not permitted 
to increase its value at any point in the course of the auction. Reductions can be made 
one time, or multiple times during the pendency of an auction. s ? can even be reduced 
to zero. 

In other words, if a seller who begins an auction with strict standards decides 
20 to open up the bidding to users with lesser Expertise or Regard, as the case may be, 
the door is opened to bids from any users that meet the lower standard. The seller will 
not be permitted to discriminate among users according to any standard other than the 
buyer's Expertise or Regard, applied consistently. In alternative embodiments, the 
user's Expertise or Regard may be monitored during the pendency of the auction and, 
25 if the value falls below the current level of s " , the selling user given the option (not 
an obligation) of releasing the user's binding bid. 



17. Data structures 

Fig. 10 shows an example of a small data set in accordance with an 
30 embodiment of the Basic High Regard method. This data set is intentionally small for 
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the sake of example. It will be understood that actual data sets usually used with the 
methods and systems shown here can be very large. In the example, there are five 
users 1002, each having a user id, such as an e-mail address. Various ones of the 
various users have rated items having item ids 1-13. In the example every user has 
authored items. Ratings received from the users for various items vary between a 
lowest rating of 0.03 (items item 7 rated by user 1 and item 4 rated by user 5) and a 
highest rating of 0.95 (items 2 and 3 rated by user 1). Applying the Basic High 
Regard method discussed above yields the Regard (HR) values shown in table 1006 
for each of users 1-5. It will be understood that the Regard values shown in this 
example are for purposes of example only and are not to be taken in a limiting sense. 

Fig. 1 1 shows an example of a data structure used to store and retrieve data 
required to perform calculations of Regard and Quality in a preferred embodiment of 
the present invention. This implementation uses circular linked lists to represent the 
sparse matrixes used to store the ratings contributed by the users. The Figure shows a 
Users linked list 1 100 and an Items linked list 1 140. Users list 1 100 contains an entry 
for p+1 users. The users have user IDs (uids) 0 through p. Items list 1 140 contains an 
entry for n+1 items that were contributed by the users. The items have item IDs (iids) 
0 through n. 

Each element in Users list 1 100 has a corresponding Authors linked list 1 120. 
In the Figure, only one Author list is shown to improve the clarity of the Figure. The 
entry in Users list 1 100 for user 2 has an associated Author listl 1 120 containing three 
entries. These entries in the Author list 1120 shown in the Figure represent the 
authors whose items have been rated by user 2. Thus, the first entry 1 122 represents 
user 1, whose item(s) were rated by user 2; the second entry 1124 represents user 4, 
whose item(s) were rated by user 2; and the third entry 1 126 represents user 7, whose 
item(s) were rated by user 2. 

Each element in an Authors list 1 120 has a corresponding Ratings linked list 
1130. In the Figure, only two Ratings list are shown for the sake of clarity. For 
example, the entry 1122 in Authors list 1120 for Author 1 has an associated Ratings 
list 1130 containing three entries. These entries in the Ratings lists 1130 represent 
items contributed by User 1 (and rated by user 2). Thus, the first entry 1132 
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represents item 3, which was contributed by user 1 and rated by user 2; the second 
entry 1 134 represents item 8, which was contributed by user 1 and rated by user 2; the 
third entry 1136 represents item 17, which was contributed by user 1 and rated by user 
2. 

Each entry in a Ratings list 1130 includes a "g" value, an author ID of the 
author (aid = author's uid), a "k" value and an "m" value. Each "g" value in a Ratings 
list 1130 represents a rating for the item. The "r" value in each entry of Authors list 
1120 is the sum of the ratings g from its corresponding Ratings list 1130. Thus, for 
example, the sum of the ratings g in entries 1132, 1134, 1136 is stored as value r in 
entry 1 122. 

Each entry in a Ratings list 1130 has a "k" value. In the example, k=l, 
although k can also have other values. The "t" value in each entry of Authors list 
1 120 is the sum of the "k" values from its corresponding ratings list 1 130. Thus, for 
example, the sum of the "k" values in entries 1132, 1134 and 1136 is stored as value 
"t" in entry 1 122. 

In the described embodiment, when author 2 rates an item of author 1, the 
rating is added to the Ratings list 1130 for Author 1 and the "t" value for Author 1 is 
incremented. 

The Figure includes a plurality of circular linked lists 1150 (used to find high 
Regard values) and a plurality of circular linked lists 1160 (used to find "Quality"). 
Only one of each of lists 1150 and 1160 is shown for the sake of clarity. Each list 
1150 is formed by a series of b_next links and represents the "r" and "t" values of a 
single author. Thus, in the Figure, list 1 150 contains entries whenever items authored 
by user 1 were rated by another user. Thus, entry 1122 represents items of user 1 
rated by user 2. Entry 1154 represents items of user 1 rated by user 0. In the 
example, entry 1 154 points to the entry for user 1 in Users list 1 100, which points in 
turn to entry 1 122, forming circular linked list 1 150. To calculate "high Regard," the 
method follows the b_next chain for a list 1150 to pick up the non-zero "r" and "t" 
values for a particular author. 

Each list 1160 is formed by a series of s_next links and represents the ratings 
"g" of a single item that has been rated by various users. Thus, in the Figure, list 1 160 
contains entries whenever item 2 of user 4 was rated by another user. Thus, entry 
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1162 represents item 2 of user 4 rated by user 2. Entry 1164 represents item 2 of user 
4 rated by user 9. In the example, entry 1164 points to entry 1142 in Items list 1140 
(for item 2). Entry 1142 points to entry 1162 , forming a circular linked list. To 
calculate "Quality" for an item, the method follows the s_next chain for a list 1160 to 
pick up the non-zero ratings "g" values. 

As shown, each user in User list 1100 has a corresponding "Regard" in high 
Regard list 170. Each entry in the lists is preferably time-stamped (to aid, for 
example, in the decay method discussed above). 

There may be situations where different types of items co-exist. For example, 
discussion group postings, participation in an online chat session, the products, 
services and assets one offers to sell in online auctions and web link 
recommendations, etc. can be considered items of different types. The concept of 
"multi-type High Regard" is created so that an item of one type may have more (or 
less) influence on the High Regard values than an item of a different type. 

18. Example Architectures 

It will be understood that the example architectures discussed herein are 
provided by way of example and are not to be taken in a limiting sense. There are a 
large number of possible architectures and network structures that benefit from having 
a server such as rating server 140 of Fig. 1 therein (or a similar rating server returning 
Expertise and Quality). In the figures that follow, the function of receiving ratings 
and user information and returning Quality and/or Regard (Expertise) information is 
preferably performed by rating server 1202. It will also be understood that rating 
server 1202 can be part of a larger data processing system performing one or more of 
the functions discussed herein. Rating server 1202 is shown separately, but is not 
required to be separate from the entities to which it provides information. In addition, 
the functions of rating server 1202 and the other entities discussed below can be 
distributed over more than one data processing system or network without departing 
from the spirit and scope of the present invention. 

Fig. 12(a) is a block diagram of a first example forum server application 
including a rating server 1202. A forum server 1204 provides, for example, the 
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content of a forum such as that shown in Fig. 2(a). Forum server 1204 accesses a 
database 1206 storing the items displayed by the forum and preferably caching the 
Quality and Regard values returned from rating server 1202, although not all servers 
1204 cache. Expertise and/or Caliber values might also be returned. Rating server 
1202 accesses its own database 1208 containing information about the ratings, Quality 
and the Regard of the various users (including associated Regards caused by 
Vouching and Discrediting, if these features are part of the system). For example, the 
data structure of Fig. 11 is preferably stored in database 1208. It will be understood 
that the databases 1206, 1208 shown can be located at other appropriate places in the 
system without departing from the spirit and scope of the invention. In Fig. 12(a), a 
user requests forums, threads and articles within the threads and posts his own items 
through interaction with forum server 1204. Forum server 1204 interacts with rating 
server 1202 to obtain the Regard of the contributing users and the item Quality 
requested. Forum server 1204 also sends ratings contributed by users to rating server 
1202 as they are received. 

Thus, forum server 1204 identifies new items contributed by the users to rating 
server 1202. The items them selves are not necessarily sent to server 1202, but server 
1202 need to know that new items have been contributed. An item ID can be 
determined either by forum server 1204 (in which case, the item ID is preferably 
stored in conjunction with a forum ID to uniquely identify the item) or by rating 
server 1202. Forum server 1202 also identifies the existence of new authors to rating 
server 1202. Again, an author id can be established either by forum server 1204 or by 
rating server 1202. 

Forum server 1204 also identifies the existence of new users to rating server 
1202. A user id can be established either by forum server 1204 or by rating server 
1202. 

Forum server 1204 also identifies the existence of new ratings to rating server 
1204. These ratings are received by forum server 1204 from users via an interface 
such as that shown in Fig. 2. In alternative embodiments, ratings are received directly 
from the users without having to go through forum server 1204. Communication 
between servers 1202 and 1204 and between the servers and user 1201 can be 
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accomplished using any appropriate protocol and message format. For example, 
rating and forum items can be sent to a user's browser using the well-known http Web 
protocol. Similarly, information can be exchanged among the elements of Fig. 12(a) 
using any appropriate non-web protocol. 

Fig. 12(b) shows another embodiment of servers 1202 and 1204 in which 
server 1202 communicates directly with the user's browser instead of with forum 
server 1204. In this embodiment, server 1204 still sends information about new users, 
new items, new ratings, etc to rating server 1202, but requests for Quality, Expertise, 
Caliber and/or Regard values are sent by a browser of user 1201 and returned by 
rating server 1202 directly to the user's browser. The browser may cache these values 
in certain embodiments. 

Fig. 12(c) shows an example of a web page where the html of the page cause a 
web page including items (or descriptions of items) to be fetched from forum server 
1204 and Regard/Expertise/Caliber and/or Quality values to be fetched separately 
from server 1202. Alternately (not shown) a web page could be fetched from a third 
server. This third party web page might include links to both forum server 1204 and 
1202. When the browser encounters these links on the web page, it requests 
information from the specified server and incorporates it into the displayed web page. 

Fig. 13 is a block diagram of another example forum server application 
communicating with a separate integrated content server 1302 and a rating server 
1202. Integrated content server 1302 communicates with forum server 1204 to obtain 
items (e.g., threads of forum messages) and communicates with forum server 1204 
(and indirectly with rating server 1202) to obtain Regard/Expertise for users and/or 
Quality for items. Server 1302 preferably also communicates with a global network. 
For example, integrated content server 1302 might obtain content from an outside 
source (for example, an online newspaper) and add ratings information to the content 
so obtained. 

Fig. 14 is a block diagram of another example forum server application 1204 
communicating with a separate integrated content server 1302 and a rating server 
1202. An ad server 1402 can be, for example, a known commercial ad server, such as 
the Doubleclick, 24/7 or Adforce ad servers. Ad server 1402 communicates with a 
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user's browser, for example, to deliver ad content. The user 1401 views the contents 
of a web page including ads from ad server 1404 and integrated content from server 
1302. The integrated content includes data from forum server 1204 and Regard 
and/or Quality values from rating server 1202. The ads can include clickthrough 
banners from ad server 1402 and/or advertiser's web site 1404. Servers 1402/1404 
access a commercial data server 1406. Commercial data server 1406 can track the 
user's identity through information provided from servers 1402 and 1404 and obtains 
the Regard and/or item Quality from rating server 1202. Commercial data server 
1406 then provides the obtained ratings information to the requesting advertiser web 
site 1404, ad server 1402 and forum server 1204 to help them target their advertising. 
Thus, rating server 1202 provides data both to the forum server 1204 and to 
commercial data server 1406, which forwards the data to its own requesting clients. 

Alternatively, rating server 1202 can provide ratings directly to the browser of 
user 1401 in a manner similar to that shown in Fig. 12(b). 

Fig. 15(a) is a flow diagram showing communication between elements of Fig. 
14 during forum/thread index intercommunication. This flow is similar to that shown 
in Fig. 13, except that Fig. 13 does not include an ad server that provides the user with 
ads on the web pages the user is viewing. Details of this process are shown in Figs. 
16(a)-16(c). 

Fig. 15(b) is a flow diagram showing communication between elements of Fig. 
14 during article view intercommunication. Here, the user requests a particular 
article/item in a current thread. The forum server requests ratings for the article from 
rating server 1202 and provides the items and ratings to the user. When the user rates 
an item, the rating is sent to rating server 1202, which calculates a new Quality for the 
item and sends the new Quality to the forum server and web server with instructions 
that the new Quality invalidates any previously cached Quality for the item. Details 
of this process are shown in Figs. 16(d)-16(f). 

Fig. 15(c) is a flow diagram showing communication between elements of Fig. 
14 during post/reply intercommunication. When the user posts a new article in a 
thread, the article/item is sent to rating server 1202. Rating server 1202 sends a 
message to web server 1302 to invalidate any cached Caliber values for the thread (as 
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opposed to items in the thread). The forum server may handle the caliber calculation. 
Alternatively, caliber may be calculated by a java applet on the user's own machine. 
Details of this process are shown in Figs. 16(g)-16(h). 

Fig. 17 is a block diagram of an example integrated content server 1302, 
where the user's browser also communicates with an e-commerce web site 1702 and 
an e-commerce sourcing server 1704. Rating server 1202 provides Quality and/or 
Regard values to forum server 1204 and to commercial data server 1406. The user 
visits the e-commerce web site, which communicates the user's identity to 
commercial data server 1406. As in Fig. 14, server 1406 communicates with rating 
server 1202 to obtain information about the user and passes it to its own requesting 
clients, such as site 1702, server 1704, and server 1204. 

Fig. 18 is a block diagram of an example auction server 1802, where the user's 
browser also communicates with an e-commerce web site 1702 and an e-commerce 
sourcing server 1704. Rating server 1202 provides Quality and/or Regard values to 
forum server 1204 and to commercial data server 1406. The user visits the e- 
commerce web site, which communicates the user's identity to commercial data 
server 1406. As in Fig. 14, server 1406 communicates with rating server 1202 to 
obtain information about the user and passes it to its own requesting clients, such as 
site 1702, server 1704, and server 1204. 

Alternatively, the functionality of rating server 1202 is integrated into auction 
web server 1802, so that auction server 1202 calculates Regard and Quality and 
receives ratings. Thus, auction web server 1802 can provide Quality and Regard 
information about items on the auction site and about auctions transactions as 
discussed above. Alternatively, rating server 1202 communicates Regard and/or 
Quality information to respective browsers of users 1810 

Fig. 19 is a block diagram of an example rating server 1202 for an individual 
and commercial rating service. Here, rating server 1202 communicates with browser 
or communication software of user 1902 to provide that user with ratings (Quality 
and/or Regard) of other users and their items. Rating server 1202 also communicates 
with commercial data server 1402 as described above in connection with Fig. 14. The 
commercial data server passes information concerning ratings obtained from rating 
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server 1202 to both e-commerce web sites and advertisers web sites and to still other 
individual users. Thus, rating server 1202 provides data both to individuals and to a 
commercial service in this example. 

Other embodiments will be apparent to those skilled in the art from consideration 
of the specification and practice of the invention disclosed herein. It is intended that 
the specification and examples be considered as exemplary only, with a true scope of 
the invention being indicated by the following claims and equivalents. 
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1 LA method for providing interactive evaluation of a content item disseminated over 

2 a computer network comprising the steps of: 

3 (a) disseminating a content item to a plurality of individual users of 

4 computers, wherein the content item is provided by one of said users; 

5 (b) receiving evaluations of the content item from the individual users; and 

6 (c) assigning a quality rating to the content item based on weightings of the 

7 evaluations provided by the individual users. 

1 2. The method of claim 1, wherein the evaluation provided by a first individual user 

2 is weighted to reflect an individual expertise rating of the first individual user. 

1 3. The method of claim 2, wherein the individual expertise of the first individual 

2 user is based on weighted evaluations by other individual users of at least one of 

3 the content items or evaluations provided by the first individual user. 

1 4. The method of claim 1, further comprising the step of sorting content items by 

2 quality rating. 
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1 5. The method of claim 2, further comprising the step of sorting content items by the 

2 individual expertise of the provider of the content item. 

1 6. The method of claim 4, wherein the evaluation provided by a first individual user 

2 is weighted to reflect an individual expertise rating of the first individual user. 

1 7. The method of claim 6, wherein the individual expertise of the first individual 

2 user is based on evaluations by other individual users of at least one of the content 

3 items or evaluations provided by the first individual user. 

1 8. The method of claim 5, further comprising the step of sorting content items by the 

2 individual expertise of the provider of the content item. 

1 9. The method of claim 2, wherein a first individual user may associate his expertise 

2 for or against a content item provided by another individual user, thereby 

3 affecting the expertise associated with the content item. 

1 1 0. The method of claim 3, wherein a first individual user may associate his expertise 

2 for or against a content item provided by another individual user, thereby 

3 affecting the expertise associated with the content item. 
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1 11. The method of claim 4, wherein a first individual user may associate his expertise 

2 for or against a content item provided by another individual user, thereby 

3 affecting the expertise associated with the content item. 

1 12. The method of claim 1, further comprising the step of revising the weightings of 

2 evaluations provided by users in accordance with pre-established criteria. 

1 13. The method of claim 12, wherein the evaluation provided by a first individual 

2 user is weighted to reflect an individual expertise rating of the first individual 

3 user. 

1 14. The method of claim 12, wherein the individual expertise of the first individual 

2 user is based on evaluations by other individual users of at least one of the content 

3 items or evaluations provided by the first individual user. 

1 15. The method of claim 13, further comprising the step of revising the individual 

2 expertise rating of the first individual user in accordance with pre-established 

3 criteria. 
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1 16. The method of claim 1, wherein an individual user navigates through information 

2 available over the network at least in part by providing evaluations of content 

3 items. 

1 17. The method of claim 3, wherein an individual user navigates through information 

2 available over the network at least in part by providing evaluations of content 

3 items. 

1 18. The method of claim 4, wherein an individual user navigates through information 

2 available over the network at least in part by providing evaluations of content 

3 items. 

1 19. The method of claim 7, wherein an individual user navigates through information 

2 available over the network at least in part by providing evaluations of content 

3 items. 

1 20. The method of claim 12, wherein an individual user navigates through 

2 information available over the network at least in part by providing evaluations of 

3 content items. 
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The method of claim 15, wherein an individual user navigates through 
information available over the network at least in part by providing evaluations of 
content items. 



Method and System for Rating Users, User-Contributed Items, 
Groupings of Related User-Contributed Items and Other Items 

5 ABSTRACT 

A method and system for constructing, applying and distributing ratings of 
users, user-contributed items, groupings of related user-contributed items and other 
items in a network environment. Users have an opportunity to rate each instance of 

10 one user interacting with other users, referred to herein as an "item." Ratings of items 
contributed by or associated with a particular user are used to construct measurements 
of the user's competence and credibility as a participant in a networked environment, 
called herein "Expertise" and "Regard." Either Expertise or Regard can be used as a 
factor in determining the relative weight assigned to each user's evaluations of items 

1 5 contributed by or associated with other users, producing a measure of the relevance, 
accuracy and importance of a particular item, referred to herein as item "Quality." For 
example, the Expertise or Regard of users rating a particular discussion group posting 
might be used to weight their ratings in a calculation of the posting' s Quality. The 
described embodiments of applicants' invention also include methods enabling users 

20 to "Vouch" for and "Discredit" items contributed by another user by associating their 
own Expertise or Regard with such items. Certain embodiments calculate Caliber, 
which is a measure of groupings of user contributed items. Certain embodiments will 
construct an environment in which users provide ratings as a natural part of navigating 
through items. 

25 
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Sample User Data 


uid 


User 


1 


gordon@cole.com 


2 


larry@marso.com 


3 


dale@cooper.com 


4 


laura@palmer.com 


5 


brian@litzinger.com 



Sample Rating Log 



uid 


aid 


iid 


rating 


1 


2 


1 


0.74 


1 


3 


2 


0.95 


1 


3 


3 


0.95 


1 


4 


4 


0.71 


1 


4 


5 


0.71 


1 


4 


6 


0.71 


1 


5 


.7 


0.03 


2 


1 


8 


0.13 


2 


1 


9 


0.13 


2 


5 


7 


0.39 


3 


1 


8 


0.27 


3 


1 


9 


0.27 


3 


4 


4 


0.065 


3 


4 


5 


0.065 


3 


5 


7 


0.795 


3 


5 


10 


0.795 


4 


2 


1 


0.92 


4 


5 


7 


0.65 


4 


5 


11 


0.65 


5 


1 


8 


0.59 


5 


2 


1 


0.44666667 


5 


2 


12 


0.44666667 


5 


2 


13 


0.44666667 


5 


3 


2 


0.91 


5 


4 


4 


0.03 



Sample Output of High Regard 


uid 


User 


HR 


1 


gordon@cole.com 


0.285943744159 


2 


larry@marso.com 


0.525147620373 


3 


dalc@cooper.com 


0.928946150922 


4 


iaura@palmer.com 


0.223508895679 


5 


b ri an @ 1 i tzi n ger.com 


0.635736019429 



uid = identifier of person doing the rating 
aid = identifier of author of item being rated 
iid = idenfitier of item being rated 
rating = rating assigned to item iid by user uid 
HR = High Regard 
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Webserver asks RatingServer 
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no 



RatingServer 
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RatingServer 
Return User ID to Webserver 




User Forum Selection 



User selects Forum 



Webserver Determines if it 
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copy of requested Forum Index 
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deliver forum index 

1 

j ForumServer Del ermines if it 
I has valid locally cached 

copy of requested Forum Index 




ForumServer constructs Forum Index 
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Injects Ad References as Required 
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| Forum Server Returns Forum — 
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1 

Webserver Returns Forum ^_ 
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I -, 

UscrBrowser contacts AdServer 
as described by Ad References 

1 

AdServer contacts RatingServer and 
returns User targeted Ad to UscrBrowser 
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1 




ForumServer Determines if it 
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Valid 
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Thread 
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Injects Ad References as 


Required 
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Thread to UserApp 




User Thread Viewing 
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Article of Thread 
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of Articles are sent to 
RatingServer 
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cache related to current thread 
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Webserver and invalidates 
cache related to current thread 




User Post 



Display User Post Dialog 
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